Batch Import

In some cases, due to the large number of elements (items, users or user-item interactions), it may be desirable to add all of them in a batch, rather than one by one. The batch endpoint allows the ability to add an arbitrary number of elements from a single file.

There are six possible statuses for the batch process:

StatusDescription
queuedThe batch upload request has been logged, but it has not been processed yet.
uploadingThe batch upload request is being processed.
canceledThe batch upload request has been canceled at the request of the user.
errorAn error occurred while processing the batch upload request. The elements were not uploaded.
completedwitherrorsThe batch upload did not complete successfully. Some of the elements were not uploaded.
completedThe batch upload completed successfully. All the elements are available.

When the batch import is requested for the first time (see Upload), the API automatically assigns queued status. To check the updated status, please refer to Get status.

Upload

Items

Upload the URL and additional info about the file that contains the items to be imported.

URL
POST /{tableId}/items/batch

Parameters

URL parameters:

ParameterMandatoryTypeDescription
tableIdStringID of the table

Fields to be included in the request body:

FieldMandatoryTypeDescription
urlStringURL to access the file that contains the items to be added
formatStringFile format: csv or json

File Format
The files must be encoded using UTF-8. There are two options: CSV or JSON.

CSV
The first row must contain the list of attribute names. The string values have to be enclosed by quotation marks (").

Example:

"itemId","name","quantity","origin","price"
"item01","apple",10,"France",1.75
"item02","orange",7,"Spain",0.98
"item03","banana",12,"Colombia",0.76

JSON
A list of items with all attributes in JSON format.

Example:

[
    {
        "itemId": "item01",
        "name": "apple",
        "quantity": 10,
        "origin": "France",
        "price": 1.75
    },
    {
        "itemId": "item02",
        "name": "orange",
        "quantity": 7,
        "origin": "Spain",
        "price": 0.98
    },
    {
        "itemId": "item03",
        "name": "banana",
        "quantity": 12,
        "origin": "Colombia",
        "price": 0.76
    }
]

Response

CodeName
202Accepted
400Bad Request
401Unauthorized
404Not Found
500Internal Server Error
503Service Unavailable

JSON that includes the ID and the status of the request.

Example
Upload a CSV file from https://www.sherpa.ai/items_file.csv. The CSV file should be stored at the URL provided, so that it can be found by the system during the uploading process.

curl -X POST "https://recom.sherpa.ai/v2/recomm/products/items/batch" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******" \
-d  "{\"url\":\"https://www.sherpa.ai/items_file.csv\",\"format\":\"csv\"}"
{
    "requestId": "d0681fb2-5d99-47f5-b4b1-eff31807376b",
    "status": "queued"
}

Users

Upload the URL and additional info about the file that contains the users to be imported.

URL
POST /users/batch

Parameters

Fields to be included in the request body:

FieldMandatoryTypeDescription
urlStringURL to access the file that contains the users to be added
formatStringFile format: csv or json

File Format
The files must be encoded using UTF-8. There are two options: CSV or JSON.

CSV
The first row must contain the list of attribute names. The string values have to be enclosed by quotation marks (").

Example:

"userId","name","interests","age"
"user01","Katherine Johnson","science,technology,travel",35
"user02","Ingrid Daubechies","technology,education",66

JSON
A list of users with all attributes in JSON format.

Example:

[
    {
        "userId": "item01",
        "name": "Katherine Johnson",
        "interests": ["science","technology","travel"],
        "age": 35
    },
    {
        "userId": "item02",
        "name": "Ingrid Daubechies",
        "interests": ["technology","education"],
        "age": 66
    }
]

Response

CodeName
202Accepted
400Bad Request
401Unauthorized
500Internal Server Error
503Service Unavailable

JSON that includes the ID and the status of the request.

Example
Upload a CSV file from https://www.sherpa.ai/users_file.csv. The CSV file should be stored at the URL provided, so that it can be found by the system during the uploading process.

curl -X POST "https://recom.sherpa.ai/v2/recomm/users/batch?url=https://www.sherpa.ai/users_file.csv&format=csv" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******" \
-d  "{\"url\":\"https://www.sherpa.ai/users_file.csv\",\"format\":\"csv\"}"
{
    "requestId": "31363c19-2913-4b99-8942-28f0a7ef1bc9",
    "status": "queued"
}

User-Item Interactions

Upload the URL and additional info about the file that contains the user-item interactions to be imported.

URL
POST /{tableId}/interactions/batch

Parameters

URL parameters:

ParameterMandatoryTypeDescription
tableIdStringID of the table

Fields to be included in the request body:

FieldMandatoryTypeDescription
urlStringURL to access the file that contains the user-item interactions to be added
formatStringFile format: csv or json

File Format
The files must be encoded using UTF-8. There are two options: CSV or JSON.

CSV
The first row must contain the list of attribute names. The string values have to be enclosed by quotation marks (").

Example:

"userId","itemId","timestamp","interactionName","value"
"user01","item01",1581681280254,"givestars",1
"user01","item03",1581680282301,"givestars",5
"user02","item02",1581673287916,"givestars",4

JSON
A list of user-item interactions with all attributes in JSON format.

Example:

[
    {
        "userId": "user01",
        "itemId": "item01",
        "timestamp": 1581681280254,
        "interactionName": "givestars",
        "value": 1
    },
    {
        "userId": "user01",
        "itemId": "item03",
        "timestamp": 1581680282301,
        "interactionName": "givestars",
        "value": 5
    },
    {
        "userId": "user02",
        "itemId": "item02",
        "timestamp": 1581673287916,
        "interactionName": "givestars",
        "value": 4
    }
]

Response

CodeName
202Accepted
400Bad Request
401Unauthorized
404Not Found
500Internal Server Error
503Service Unavailable

JSON that includes the ID and the status of the request.

Example
Upload a CSV file from https://www.sherpa.ai/interactions_file.csv. The CSV file should be stored at the URL provided, so that it can be found by the system during the uploading process.

curl -X POST "https://recom.sherpa.ai/v2/recomm/products/interactions/batch" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******" \
-d  "{\"url\":\"https://www.sherpa.ai/interactions_file.csv\",\"format\":\"csv\"}"
{
    "requestId": "c225dc3f-10bf-42bf-b544-bd1ce953bdab",
    "status": "queued"
}

Get Upload Status

Items

Return the status of a previous batch request for items.

URL
GET /{tableId}/items/batch/{requestId}

Parameters

URL parameters:

ParameterMandatoryTypeDescription
tableIdStringID of the table
requestIdStringID returned by a previous batch request

Response

CodeName
200OK
400Bad Request
401Unauthorized
404Not Found
429Too Many Requests
500Internal Server Error
503Service Unavailable

JSON that includes the current status of the request and a list of IDs of the items that could not be inserted, due to an error.

Example
Get the current status of the request d0681fb2-5d99-47f5-b4b1-eff31807376b.

curl -X GET "https://recom.sherpa.ai/v2/recomm/products/items/batch/d0681fb2-5d99-47f5-b4b1-eff31807376b" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "d0681fb2-5d99-47f5-b4b1-eff31807376b",
    "status": "completed"
}

Items Not Added
In the event of an error during the insertion of an item, the loading engine will skip it and log it as an item not added. In this case, when the loading process completes, the upload task will be flagged as completed_with_errors and a list of the items that were not added will be returned as itemsNotAdded.

Example
Get the current status of the request d0681fb2-5d99-47f5-b4b1-eff31807376b.

curl -X GET "https://recom.sherpa.ai/v2/recomm/products/items/batch/d0681fb2-5d99-47f5-b4b1-eff31807376b" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "d0681fb2-5d99-47f5-b4b1-eff31807376b",
    "status": "completed_with_errors",
    "itemsNotAdded": [
        {
            "errorId":"item348",
            "code": "EntityAlreadyExistsException",
            "message":"Item [item348] could not be created because it already exists."
        },
        {
            "errorId":"item705",
            "code": "InvalidArgumentException",
            "message":"Attribute price with value [high] is not of double type."
        }
    ]
}

Users

Return the status of a previous batch request for users.

URL
GET /users/batch/{requestId}

Parameters

URL parameters:

ParameterMandatoryTypeDescription
requestIdStringID returned by a previous batch request

Response

CodeName
200OK
400Bad Request
401Unauthorized
404Not Found
429Too Many Requests
500Internal Server Error
503Service Unavailable

JSON that includes the current status of the request and a list of IDs of the users that could not be inserted, due to an error.

Example
Get the current status of the request 31363c19-2913-4b99-8942-28f0a7ef1bc9.

curl -X GET "https://recom.sherpa.ai/v2/recomm/products/items/batch/31363c19-2913-4b99-8942-28f0a7ef1bc9" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "31363c19-2913-4b99-8942-28f0a7ef1bc9",
    "status": "completed"
}

Users Not Added
In the event of an error during the insertion of a user, the loading engine will skip it and log it as a user not added. In this case, when the loading process completes, the upload task will be flagged as completed_with_errors and a list of the users that were not added will be returned as usersNotAdded.

Example
Get the current status of the request 31363c19-2913-4b99-8942-28f0a7ef1bc9.

curl -X GET "https://recom.sherpa.ai/v2/recomm/users/batch/31363c19-2913-4b99-8942-28f0a7ef1bc9" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "31363c19-2913-4b99-8942-28f0a7ef1bc9",
    "status": "completed_with_errors",
    "usersNotAdded": [
        {
            "errorId":"user816",
            "code": "EntityAlreadyExistsException",
            "message":"User [user816] could not be created because it already exists."
        },
        {
            "errorId":"user331",
            "code": "InvalidArgumentException",
            "message":"Attribute age with value [old] is not of integer type."
        }
    ]
}

User-Item Interactions

Return the status of a previous batch request for user-item interactions.

URL
GET /{tableId}/interactions/batch/{requestId}

Parameters

URL parameters:

ParameterMandatoryTypeDescription
tableIdStringID of the table
requestIdStringID returned by a previous batch request

Response

CodeName
200OK
400Bad Request
401Unauthorized
404Not Found
429Too Many Requests
500Internal Server Error
503Service Unavailable

JSON that includes the current status of the request and a list of IDs of the user-item interactions that could not be inserted, due to an error.

Example
Get the current status of the request c225dc3f-10bf-42bf-b544-bd1ce953bdab.

curl -X GET "https://recom.sherpa.ai/v2/recomm/products/interactions/batch/c225dc3f-10bf-42bf-b544-bd1ce953bdab" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "c225dc3f-10bf-42bf-b544-bd1ce953bdab",
    "status": "completed",
}

User-Item Interactions Not Added
In the event of an error during the insertion of a user-item interaction, the loading engine will skip it and log it as a user-item interaction not added. In this case, when the loading process completes, the upload task will be flagged as completed_with_errors and a list of the user-item interactions that were not added will be returned as interactionsNotAdded.

Example
Get the current status of the request c225dc3f-10bf-42bf-b544-bd1ce953bdab.

curl -X GET "https://recom.sherpa.ai/v2/recomm/products/interactions/batch/c225dc3f-10bf-42bf-b544-bd1ce953bdab" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "c225dc3f-10bf-42bf-b544-bd1ce953bdab",
    "status": "completed_with_errors",
    "interactionsNotAdded": [
        {
            "errorId":"user01/products/item03/givestars/1581673287916",
            "code": "EntityAlreadyExistsException",
            "message":"Interaction [givestars] on Item [item03] of User [user01] at time [1581673287916] could not be created because it already exists."
        }
    ]
}

Cancel Upload

Items

Cancel a previous batch request for items. This can only be done when the status of the request is queued.

URL
DELETE /{tableId}/items/batch/{requestId}

Parameters

URL parameters:

ParameterMandatoryTypeDescription
tableIdStringID of the table
requestIdStringID returned by a previous batch request

Response

CodeName
200OK
400Bad Request
401Unauthorized
404Not Found
500Internal Server Error
503Service Unavailable

JSON that includes the current status of the request.

Example
Cancel the request d0681fb2-5d99-47f5-b4b1-eff31807376b.

curl -X DELETE "https://recom.sherpa.ai/v2/recomm/products/items/batch/d0681fb2-5d99-47f5-b4b1-eff31807376b" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "d0681fb2-5d99-47f5-b4b1-eff31807376b",
    "status": "canceled"
}

Users

Cancel a previous batch request for users. This can only be done when the status of the request is queued.

URL
DELETE /users/batch/{requestId}

Parameters

URL parameters:

ParameterMandatoryTypeDescription
requestIdStringID returned by a previous batch request

Response

CodeName
200OK
400Bad Request
401Unauthorized
404Not Found
500Internal Server Error
503Service Unavailable

JSON that includes the current status of the request.

Example
Cancel the request 31363c19-2913-4b99-8942-28f0a7ef1bc9.

curl -X DELETE "https://recom.sherpa.ai/v2/recomm/products/items/batch/31363c19-2913-4b99-8942-28f0a7ef1bc9" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "31363c19-2913-4b99-8942-28f0a7ef1bc9",
    "status": "canceled"
}

User-Item Interactions

Cancel a previous batch request for user-item interactions. This can only be done when the status of the request is queued.

URL
DELETE /{tableId}/interactions/batch/{requestId}

Parameters

URL parameters:

ParameterMandatoryTypeDescription
tableIdStringID of the table
requestIdStringID returned by a previous batch request

Response

CodeName
200OK
400Bad Request
401Unauthorized
404Not Found
500Internal Server Error
503Service Unavailable

JSON that includes the current status of the request.

Example
Cancel the request c225dc3f-10bf-42bf-b544-bd1ce953bdab.

curl -X DELETE "https://recom.sherpa.ai/v2/recomm/products/interactions/batch/c225dc3f-10bf-42bf-b544-bd1ce953bdab" \
-H  "Content-Type: application/json" \
-H  "X-Sherpa-apikey: XXXX-SHERPA-DELIVERED-PUBLIC-APIKEY-XXXX" \
-H  "X-Sherpa-timestamp: 1548084514112" \
-H  "X-Sherpa-nonce: XXXX-SHERPA-RANDOM-UUID-XXXX" \
-H  "X-Sherpa-hmac: xxxxyyyyyyy***signature******"
{
    "requestId": "c225dc3f-10bf-42bf-b544-bd1ce953bdab",
    "status": "canceled",
}