MongoDB

​

What is it?

MongoDB is a popular open-source NoSQL (non-relational) database management system. It is designed to handle large volumes of unstructured or semi-structured data, making it well-suited for applications with rapidly evolving schemas or complex data models. MongoDB stores data in flexible, JSON-like BSON (Binary JSON) documents, and its architecture allows for horizontal scaling, enabling efficient handling of growing datasets. It supports dynamic queries, indexing, and provides features like replication and sharding for high availability and scalability. MongoDB is commonly used in modern web development and other scenarios where flexible and scalable data storage is essential.

The purpose of integration is that peaq storage does not store big data. It should be handled by your off-chain storage. On-chain storage data should be small so it doesn’t delay execution.

​

On-Chain Storage

Smart Contracts: On-chain storage primarily involves the use of smart contracts. Smart contracts are self-executing contracts with the terms of the agreement directly written into code. They reside on the blockchain and can store small amounts of data.

State Variables: State variables are used to store persistent data. These variables hold information that needs to be preserved across transactions and blocks.

Immutable Ledger: The blockchain itself serves as an immutable ledger that records the history of all transactions. Each block contains a reference to the previous block, creating a chain of blocks that cannot be altered without changing all subsequent blocks.

​

Off-chain Storage:

Databases: Off-chain storage involves the use of traditional databases or distributed databases to store large amounts of data that do not need to be recorded on the blockchain itself.

Decentralized Storage Networks: Some blockchain projects leverage decentralized storage networks like IPFS (InterPlanetary File System) or Swarm. These networks store data across multiple nodes, providing a decentralized and distributed storage solution.

Oracles: Off-chain data can also be brought on-chain through oracles. Oracles are entities or services that provide external information to smart contracts or programs running on-chain, enabling them to interact with off-chain data.

​

Architectural Flow:

Data Generation: Data is generated either on-chain through transactions or off-chain through external sources.

On-chain Storage: Relevant data is stored on-chain using smart contracts and state variables. This data is typically small and critical for the execution of on-chain logic and external services business logic that require data to be immutable.

Off-chain Storage: Large or less critical data is stored off-chain in databases or decentralized storage networks. This data may include files, images, or other information not necessary for the consensus mechanism.

Interaction: Smart contracts may interact with off-chain data using oracles, fetching external information and incorporating it into their logic. Or external services may interact with on-chain data to execute essential business logic.

Immutable Record: All on-chain data is recorded in blocks on the blockchain, creating an immutable and transparent record of transactions and changes to on-chain state.

​

Integration tutorial

For data verification and immutability checks, you can store the hash of the data being stored off-chain on our chain (in the DID Document/peaq storage). By doing this, you can fetch the hash from the chain and use it to verify the data being fetched from your off-chain storage. You can also map the keys of the on-chain key/value pair with the off-chain storage key. Below is an example code of using our peaqStorage with MongoDB off-chain storage:

​

1. Install the required packages:

npm install mongodb @polkadot/api

​

2. Create a tsconfig.json file in your project directory with the following configuration:

{
  "compilerOptions": {
    "target": "es2018",
    "module": "commonjs",
    "strict": true,
    "esModuleInterop": true
  }
}

​

3. Create a file named app.ts with the following code:

import { MongoClient, Db, Collection, ObjectId } from 'mongodb';

class MongoDBStorage {
  private client: MongoClient;
  private db!: Db;
  private collection!: Collection;
  private collectionName: string;
  private dbName: string;

  constructor() {
    const connectionString = 'mongodb://localhost:27017';
    this.client = new MongoClient(connectionString);
    this.dbName = 'yourDataBaseName';
    this.collectionName = 'yourCollectionName';
  }

  async connect() {
    try {
      await this.client.connect();
      console.log('Connected successfully to MongoDB server');
      this.db = this.client.db(this.dbName);
      this.collection = this.db.collection(this.collectionName);
    } catch (error) {
      console.error('Error connecting to MongoDB:', error);
      throw error;
    }
  }

  async disconnect() {
    try {
      await this.client.close();
      console.log('Disconnected successfully from MongoDB server');
    } catch (error) {
      console.error('Error disconnecting from MongoDB:', error);
      throw error;
    }
  }

  async insertDocument(document: any) {
    try {
      const result = await this.collection.insertOne(document);
      console.log('Document inserted:', result);
      return result;
    } catch (error) {
      console.error('Error inserting document:', error);
      throw error;
    }
  }

  async findDocumentById(id: string) {
    const result = await this.collection.findOne({ _id: new ObjectId(id) });
    console.log('Found document by ID:', result);
    return result;
  }

  async updateDocument(id: string, update: any) {
    const result = await this.collection.updateOne({ _id: new ObjectId(id) }, { $set: update });
    console.log('Updated document:', result.modifiedCount);
    return result.modifiedCount > 0;
  }

  async deleteDocument(id: string) {
    const result = await this.collection.deleteOne({ _id: new ObjectId(id) });
    console.log('Deleted document:', result.deletedCount);
    return result.deletedCount > 0;
  }

}

export {MongoDBStorage}

​

4. MongoDB Setup

You either have the option to deploy your MongoDB database locally (self-managed), to the cloud (using MongoDB Atlas), serverless (pay-as-you-go), or you can use the community edition.

The next step in the process is to download MongoDB to set your connection string and configure your database.

• Follow the steps to download MongoDB
• Run the cmd atlas setup to create and configure an atlas account
  • After executing the cmd your terminal will output your cluster name, cloud provider location, and display the admin username and password.
• Using the code from step 3 above, change the connection string to include your username and password. The MongoDB documentation gives an explanation on how to construct this string.
• Run the cmd tsc to compile your typescript code. Change the generated app.js file to have the new name app.cjs. This will be needed for the next step when we import the MongoDB class into the script that interacts with peaq.

​

4. Connect with peaq network:

• Create a new file in the same repository and call it peaq.js
• In package.json add "type": "module"
• Copy and paste the below code to initialize peaq network connection

import { ApiPromise, WsProvider } from '@polkadot/api';
import { MongoDBStorage } from './app.cjs';
import { u8aToHex, stringToU8a, u8aToU8a, u8aConcat } from "@polkadot/util";
import Keyring from "@polkadot/keyring";
import { decodeAddress, blake2AsHex } from '@polkadot/util-crypto';

// using agung url to connect to provider. Change to peaq when necessary
const provider = new WsProvider('wss://peaq-agung.api.onfinality.io/public-ws');
const api = await ApiPromise.create({ provider });

// creates a storage key to read data from chain
function createStorageKey(address, itemType) {
    const storageKeyByteArray = [];
    const decodedAddress = decodeAddress(address, false, 42);
    storageKeyByteArray.push(decodedAddress);

    const hashItemType = u8aToU8a(itemType);
    storageKeyByteArray.push(hashItemType);

    const key = u8aConcat(...storageKeyByteArray);

    const storageKey = blake2AsHex(key, 256);
    return storageKey
  }

// Store Big data in MongoDB, hash data on peaq Network, read to confirm data is the same
const main = async () => {
    // Data to store in MongoDB
    const data = "Hello World";

    // Store data in MongoDB
    const mongoDBStorage = new MongoDBStorage();
    await mongoDBStorage.connect();
    const insertedDocument = await mongoDBStorage.insertDocument({text: data});
    await mongoDBStorage.disconnect();

    // Hash data to store on chain
    const mongoID = insertedDocument.insertedId.toString();
    const storeHashData = u8aToHex(stringToU8a(data));

    // Create wallet from seed of a user funded wallet
    const keyring = new Keyring({ type: 'sr25519' });
    const owner = keyring.addFromUri(owner_seed);

    // Add to MongoDB id and hashed data to peaq storage
    await api.tx.peaqStorage.addItem(mongoID, storeHashData).signAndSend(owner);

    await sleep(30000); // wait 30 seconds to guarantee it has been added on chain.

    // Retrieve data from off-chain storage based on the id
    await mongoDBStorage.connect();
    const foundDocument = await mongoDBStorage.findDocumentById(mongoID);
    await mongoDBStorage.disconnect();

    const readId = foundDocument._id.toString();

    // Create a storage key and read hashed on-chain data storage
    const storageKey = createStorageKey(owner.address, readId);
    var onChainHashData = await api.query.peaqStorage.itemStore(storageKey);

    // Confirm data storage hashed value and text in mongoDB are the same
    onChainHashData = u8aToHex(onChainHashData);
    const hashData = u8aToHex(stringToU8a(foundDocument.text));
    console.log(onChainHashData == hashData);

    await api.disconnect();
  }

  function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
  }

main();

The code above creates an instance of the MongoDBStorage() Class that was created and generated in step 3. It stores the ‘big data’ in MongoDB. The hash of that data is sent to peaq storage as a pair of MongoDB_Id: Hashed_Data. The data is then retrieved from the MongoDB based on the ID and the hashed data is read, then compared, to show the immutability.

​

Summary

In this documentation, MongoDB is presented as the off-chain storage solution to complement the peaq network’s on-chain storage, which is limited to small, critical data maintained via smart contracts and state variables on an immutable ledger.