Data Integration

Modified on Tue, 12 Nov at 3:56 PM

Data Integration

Fetch Datasource Task

Fetches a datasource task from the server.

Identifier: fetch_datasource_task

Elements

  • datasource_id:Contains the ID of the data source to receive a task from.
    • Type: String
  • sheet_revision:Contains the revision of the datasource you want to use.
    • Type: Long
  • payload_variable_name:Contains the payload of the fetched task that will be converted and stored in the context with the key specified here.
    • Type: String
  • task_variable_name:Contains the task object that will be stored in the context with the key specified here.
    • Type: String
  • datasource_task_id:Contains the ID of the task that should be fetched.
    • Type: String
  • request_new:Indicates if a new and empty task should be fetched (alternative to specifying a datasource_task_id).
    • Type: Bool
  • row_count:The number of rows a new and empty task should have when requesting a new task.
    • Type: Long
  • connection_timeout:Contains the timeout in milliseconds.
    • Type: Long

Output

The event tag of this event is fetch_datasource_task.

  • OK: The datasource task has been successfully fetched.
  • ERROR: There was an error while fetching the task. If the server provides an error message, it will be found in the payload. The payload is structured as follows:
    • error
    • error.message
    • error.class
    • error.category
    • error.code
  • TIMEOUT: The server did not send a response within the specified time (connection_timeout)

Example

The fetch datasource task is typically used in conjunction with a datasource input:

<fetch_datasource_task id="fetch_datasource_task">
    <param name="datasource_id" type="string">§{ configuration.datasource.value.id }§</param>
    <param name="sheet_revision" type="string">§{ configuration.datasource.value.selectedRevision }§</param>
    <param name="payload_variable_name" type="string">ds_§{ replace configuration.datasource.value.id '-' ''  }§</param>
    <param name="task_variable_name" type="string">ds_§{ replace configuration.datasource.value.id '-' '' }§_task</param>
    <param name="request_next" type="bool">§{ #if configuration.enable_scan.value}§false§{else}§true§{/if}§</param>
    <param name="request_new" type="bool">§{ #if configuration.request_new.value}§true§{else}§false§{/if}§</param>
    <param name="row_count" type="long">§{ configuration.row_count.value }§</param>
    <param name="datasource_task_id" type="string">#{task_id}</param>
    <param name="connection_timeout" type="string">?{§{configuration.timeout.value}§*1000}?</param>
</fetch_datasource_task>

Update Datasource Task

Updates a datasource task on the server.

Identifier: update_datasource_task

Elements

  • datasource_task_id:Contains the ID of the task to be changed. This is the value of the column configured as the identifier in the datasource settings.
    • Type: String
  • rows:Contains the row object to be changed.
    • Type: Object
  • payload_variable_name:Contains the variable in which the payload of the task is stored in the context.
    • Type: String
  • connection_timeout:Contains the timeout value in milliseconds.
    • Type: Long

Output

The event tag of this event is update_datasource_task.

  • OK: The datasource task has been successfully fetched.
  • ERROR: There was an error while fetching the task. If the server provides an error message, it will be found in the payload. The payload is structured as follows: error
    • error.message
    • error.class
    • error.category
    • error.code
  • TIMEOUT: The server did not send a response within the specified time (connection_timeout)

Example

The datasource task can be used in conjunction with a datasource input. The parameters for datasource_task_id and payload_variable_name depend on where your data is saved. The example below will work with the example shown in the fetch_datasource_task input action:

<update_datasource_task id="update_ds_with_row">
    <param name="datasource_task_id" type="string">?{context.ds_§{ replace validation.datasource.value.id '-' '' }§_task.id}?</param>
    <param name="rows">#{current_row}</param>
    <param name="payload_variable_name">ds_§{ replace validation.datasource.value.id '-' ''  }§</param>
</update_datasource_task>

Update Datasource Task Status

Updates the status of a datasource task.

Identifier: update_datasource_task_status

Elements

  • datasource_task_id:Contains the ID of the task that should be fetched. This is the value of the column configured as the identifier in the datasource settings.
    • Type: String
  • status: Contains the current stage of the task (e.g., DONE, CANCELED, PAUSED, or IN_PROGRESS).
    • Type: String
  • connection_timeout:Contains the timeout value in milliseconds.
    • Type: Long

Output

The Event Tag of this event is update_datasource_task_status.

  • OK: The datasource task has been successfully fetched.
  • ERROR: There was an error while fetching the task. If the server provides an error message, it will be found in the payload. The payload is structured as follows:error
    • error.message
    • error.class
    • error.category
    • error.code
  • TIMEOUT: The server did not send a response within the specified time (connection_timeout)

Example

This example shows that this can be used in conjunction with a datasource input:

<update_datasource_task_status id="cancel_ds_task">
    <param name="datasource_task_id" type="string">?{context.ds_§{ replace validation.datasource.value.id '-' '' }§_task.id}?</param>
    <param name="status" type="string">CANCELED</param>
    <param name="connection_timeout" type="string">10000</param>
</update_datasource_task_status>

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article