Dynamic Data Manager node
Dynamic Data Manager node lets you manage the data that you want to store.
Model
You can choose the schema from this drop-down.
Action
Choose an action from the drop-down that you want to perform on the schema.
Read
Based on the parameters given in the payload, the Dynamic Data Manager node will fetch the data.
In the previous image, the Dynamic Data Manager node will fetch all the data items from the Project.
“limit” specifies the number of data items to be shown per page in the result object. The maximum value is 999. Always specify a limit. If you do not specify a limit and there are more than 999 results, it will return an invalid number of results.
“page” is the page number to be shown in the results object. The first page starts at 1.
“sortBy” is used to sort the fetched data based on the criteria.
For example, {"sortBy":{"createdAt":-1}} will sort the data items shown in the result array based on the created date displaying the latest created data item first.
The following image shows the result payload.
You can identify a data item using the ID or the Item Name. You can use either of them to get the data item in the Dynamic Data Manager node.
The ID is unique for a data item. You will get only one result when you use the correct ID.
Item Name is asked while creating the Data item manually. Multiple data items can have the same item name. You can query the data using the item name.
Example 1
The below example will return all the values that have the status as Inactive, age greater than 32, and whose page number equals the dynamic value passed.
If there are more than 999 items we need to concatenate the results.
Because we are doing a loop to get all the pages of data, it is a good idea while testing to add a delay to throttle the number of query requests, in case you make a mistake and created an infinite loop. Then either changes the delay to 1 ms or remove the delay completely.
The Dynamic Data Manager node is set to Read. The AND condition is set for status and age.
The number of documents per page is 999 items. Which is the max limit.
In sortBy
, we used createdAt
. You can also use modifiedAt
.
The createdAt
, modifiedAt
, and _id
are always returned in the collection.
{
"$and": [{
"status": "Inactive"
},
{
"age":{ "$gte":"32" }
}],
"createdAt": { "$gt" : "{{msg.startDate}}" },
"limit": 999,
"page": "{{msg.pagePntr}}",
"sortBy": {"createdAt":-1}
}
In the init function node, set up the pagePntr
to start at 1. A common mistake is to start at 0.
Also setting up the array to store the results, in this case, msg.userDataArray = [ ]
msg.pagePntr = 1; // init page pointer to 1
msg.userDataArray = []; // init aray
var moment = global.get('moment');
const now = moment(); // use moment to creat time 20 days earlier than today
const DaysAgo = now.subtract(20, "days").startOf("day").utc().toISOString();
msg.startDate = DaysAgo;
return msg;
The loop function node concatenates the results of each page until it collects all the data.
var userData = msg.data.dynamicData.result; // Get the results from dynamic data node.
if (userData.length > 0) { // is there any data.
msg.userDataArray = msg.userDataArray.concat(userData); // append to the last array.
msg.pagePntr = parseInt(msg.pagePntr) + 1; // increment the page pointer.
// node.warn('Next page ' + msg.pagePntr);
if(msg.data.dynamicData.paging.hasMore.hasNext === false) { // is there another page. if not exit loop
return [null,msg]
} else {
return [msg,null] // get the next page of data.
}
} else {
return [null,msg] // no data
}
Example 2
This example deals with the Read, OR condition, AND, NOT Equal.
You can get more query commands at https://docs.mongodb.com/manual/tutorial/query-documents/
Create
Lets you create a new data item. If you don’t specify a project ID in the payload, the data is created as Global data.
As shown in the following image, the payload may contain hard-coded values or mustache tags. Data in the mustache tag is converted to a string while saving the data.
To avoid encoded or Unicode characters getting saved, place an ampersand (&) in the mustache tag.
Creating multiple data items
You can create multiple data items at the same time in two ways.
Place multiple objects in an array. In the following image, an array with two data items is given in the payload of the Dynamic Data Manager node.
Create an array in the function node placed before the Dynamic Data Manager node. The following code lets you create a sample array and store it in msg.payload.data.
When using this method to create multiple data items, the payload in the Dynamic Data Manager node should be left empty.
Update
If the field is available in the schema, you can update the content using the payload. The _id is used to identify the data item (required item). The field that you want to update must be inside the data object.
For example,
Example 1
You can update the field value of hospitalCounty field in the schema using the below code.
In the below image, a field name hospitalCounty is available in the schema and that will be updated with the value “county”.
Example 2
In this example user, each record has a unique id called a token, only one result will be returned and we are going to update the runStatus to Complete
Example 3
In this example, we will use the function node to construct the entire payload and content to be updated and push it to the Dynamic data manager node.
Consider the schema with the Boolean and Datetime fields and you wanted to format the datetime field before updating it.
Use the function node to format the date time and construct the msg.payload.data
property as shown below.
The _id
property uniquely identifies the document and the data
object contains the schema fields to be updated.
Update the Model and Action (update). Leave the Dynamic data manager node’s payload field empty as we have constructed the payload object in the function node.
Upsert
If multiple results are found for a query, the oldest data item gets updated.
Queries fields are available
If the queried fields are available, the schema fields are updated with the content given in the payload. In the following example, queried fields are available, but one of the payload properties is not available in the schema. NewField is not available in the schema. Except for NewField, all the other fields will be updated based on the payload.
Alternatively, you can create an array in the function node placed before the Dynamic Data Manager node to send the data to msg.payload.data
.
When using this method to create multiple data items, the payload in the Dynamic Data Manager node should be left empty.
Queried fields are not available
If the queried fields are not available, it creates a data item and update the available fields in the schema based on the payload. For example, the following image shows the Newquery is not available as a field in the schema. Therefore, a new data item with a new id will be created with the payload.
Delete
You can Delete the data using a query. If the query matches multiple Data items, each of them is deleted.
Payload
A Function node should be placed prior to the Dynamic Data Manager node to verify the mandatory fields of the schema (model). These schema fields will be saved/queried based on the details given in this payload field of the Dynamic Data Manager node.
The output for the Dynamic Data Manager node will be in the msg.data object.