Aurresan eta Gomendatzeko Adimen Artifiziala

Tutorial 1: Building a Recommender System

Quickstart Guide

The goal of this tutorial is to provide detailed, step-by-step instructions to build the minimal structure for a recommender system. The examples provided in this tutorial are based on the dataset Data Science for Good: DonorsChoose.org.

We are going to build a system that recommends projects to donors with similar interests, based on previous donations. Note that we have omitted the headers required to authenticate the API user, for the sake of clarity. Please, refer to the documentation for the corresponding instructions.

Create a Catalog

This section shows how to build the structure of an item catalog. In this case, we have a CSV file containing the following information about projects from DonorsChoose:

FieldDescription
itemIdA string that uniquely identifies each project
nameThe title of the project
descriptionA description of the project, the school, etc.
needStatementThe specific requirements to accomplish the project
categoryThe area or subject the project fits into
levelThe level of the classroom
resourceThe type of requested resources
costThe total amount ($) needed to meet the needs of the project
statusThe current status of the proposal

Create a Table

First of all, we have to create a new table, called projects, in the database. You can find more information about which recommender engine to choose in Tutorial 4 and in the How It Works section.

You can also add the attributes. There are two parameters that need to be specified (see the documentation for a detailed explanation). The first, the type of attribute, is mandatory. The second, indexed, should only be used for those attributes that will be used to filter the projects.

Note: There is no need to create itemId, since it is also managed internally by the API.

POST /v2/recomm/tables HTTP/1.1

{
   "tableId":"projects",
   "engine":"content_based",
   "attributes":[
      {
         "attributeId":"name",
         "type":"string"
      },
      {
         "attributeId":"description",
         "type":"string"
      },
      {
         "attributeId":"needStatement",
         "type":"string"
      },
      {
         "attributeId":"category",
         "type":"set"
      },
      {
         "attributeId":"level",
         "type":"category",
         "indexed":true
      },
      {
         "attributeId":"resource",
         "type":"category",
         "indexed":true
      },
      {
         "attributeId":"cost",
         "type":"double"
      },
      {
         "attributeId":"status",
         "type":"category",
         "indexed":true
      }
   ]
}

If everything goes correctly, the server response should be:

HTTP/1.1 201 Created

By default, when a new table is created, in addition to the attributes defined by the user, the system creates three attributes: expiration. We can check, by using the following command:

GET /v2/recomm/projects/attributes HTTP/1.1

which returns:

HTTP/1.1 200 OK
Content-Type: application/json
[
    {
        "attributeId": "name",
        "type": "string",
        "indexed": "false"
    },
    {
        "attributeId": "description",
        "type": "string",
        "indexed": "false"
    },
    {
        "attributeId": "needStatement",
        "type": "string",
        "indexed": "false"
    },
    {
        "attributeId": "category",
        "type": "set",
        "indexed": "false"
    },
    {
        "attributeId": "level",
        "type": "category",
        "indexed": "true"
    },
    {
        "attributeId": "resource",
        "type": "category",
        "indexed": "true"
    },
    {
        "attributeId": "cost",
        "type": "double",
        "indexed": "false"
    },
    {
        "attributeId": "status",
        "type": "category",
        "indexed": "true"
    },
    {
        "attributeId": "expiration",
        "type": "timestamp",
        "indexed": "true"
    }
]

Add a New Item

Now we can add items to the catalog. Let us consider the following project:

FieldValue
itemId69da11b15b82cf59c389ba81c444731e
nameA Whole New World
descriptionSeeing my students learn is the thing that makes me the happiest. I love to be able to give them all the information that they need in order to understand and appreciate the world that they live in. Curious, creative, and very vocal, my students are the future of our society and I want to make sure they are equipped and ready to handle it. Faced with some difficult obstacles that they have to go through in their own neighborhood, I want them to know that the obstacles in the classroom can be overcome. With hard work and determination they will become the best of our future. My students truly love science. They enjoy the lessons that they do in class. An iPad can show them so much more about science. They would be able to watch videos about animals, see the different stages of matter and observe the life cycle of a plant. My students need to see first hand how science works in the world. I want them to see how science is all around them everyday by them focusing on the plants and trees, the sun and the water cycle. An iPad will show the students in real time how science affects us worldwide.
needStatementMy students need an iPad to help them learn more about science.
categoryMath & Science
levelGrades PreK-2
resourceComputers & Tablets
cost374.64
statusLive

We can add it to the database by using the following command:

POST /v2/recomm/projects/items HTTP/1.1

{
    "itemId": "69da11b15b82cf59c389ba81c444731e",
    "name": "A Whole New World",
	"description": "Seeing my students learn is the thing that makes me the happiest. I love to be able to give them all the information that they need in order to understand and appreciate the world that they live in. Curious, creative, and very vocal, my students are the future of our society and I want to make sure they are equipped and ready to handle it. Faced with some difficult obstacles that they have to go through in their own neighborhood, I want them to know that the obstacles in the classroom can be overcome. With hard work and determination they will become the best of our future. My students truly love science. They enjoy the lessons that they do in class. An iPad can show them so much more about science. They would be able to watch videos about animals, see the different stages of matter and observe the life cycle of a plant. My students need to see first hand how science works in the world. I want them to see how science is all around them everyday by them focusing on the plants and trees, the sun and the water cycle. An iPad will show the students in real time how science affects us worldwide.",
	"needStatement": "My students need an iPad to help them learn more about science.",
	"category": ["Math & Science"],
	"level": "Grades PreK-2",
	"resource": "Computers & Tablets",
	"cost": 374.64,
	"status": "Live"
}

The response is to this is:

HTTP/1.1 201 Created

This operation can be done anytime we want to include a new item in the catalog. However, the API also has an endpoint to load massive data from a single file at once. Go to Tutorial 2 for a detailed example about this option.

The following GET command

GET /v2/recomm/projects/items/69da11b15b82cf59c389ba81c444731e HTTP/1.1

confirms that the item has been correctly stored in the database:

HTTP/1.1 200 OK
Content-Type: application/json
{
    "name": "A Whole New World",
    "description": "Seeing my students learn is the thing that makes me the happiest. I love to be able to give them all the information that they need in order to understand and appreciate the world that they live in. Curious, creative, and very vocal, my students are the future of our society and I want to make sure they are equipped and ready to handle it. Faced with some difficult obstacles that they have to go through in their own neighborhood, I want them to know that the obstacles in the classroom can be overcome. With hard work and determination they will become the best of our future. My students truly love science. They enjoy the lessons that they do in class. An iPad can show them so much more about science. They would be able to watch videos about animals, see the different stages of matter and observe the life cycle of a plant. My students need to see first hand how science works in the world. I want them to see how science is all around them everyday by them focusing on the plants and trees, the sun and the water cycle. An iPad will show the students in real time how science affects us worldwide.",
    "needStatement": "My students need an iPad to help them learn more about science.",
    "category": [
        "Math & Science"
    ],
    "level": "Grades PreK-2",
    "resource": "Computers & Tablets",
    "cost": 374.64,
    "status": "Live"
}

Define the User Profile

In this section, we will explain how to define the profile of the donors that will get recommendations about projects from DonorsChoose. In this example, we are going to work with a CSV file containing the following information about the donors:

FieldDescription
userIdA string that uniquely identifies each donor
cityThe donor's city of residence
stateThe donor's state of residence
isTeacherWhether the donor is a teacher or not
zipCodeThe first three digits of the zip code of the donor's place of residence

Create User Attributes

When adding new attributes, there are two parameters that need to be specified (see documentation for a detailed explanation), the same as for item attributes: type and indexed.

Note: There is no need to create the userId attribute, since it is managed internally by the API.

In this case, we will choose city, state and isTeacher as indexable attributes. The command is the following:

POST /v2/recomm/users/attributes HTTP/1.1

{
  "attributes": [
    {
      "attributeId": "city",
      "type": "string",
      "indexed": "true"
    },
    {
      "attributeId": "state",
      "type": "category",
      "indexed": "true"
    },
    {
      "attributeId": "isTeacher",
      "type": "boolean",
      "indexed": "true"
    },
    {
      "attributeId": "zipCode",
      "type": "int"
    }
  ]
}

The response to all of them is:

HTTP/1.1 201 Created

To confirm that all attributes were added correctly, we can run the GET command:

GET /v2/recomm/users/attributes HTTP/1.1

which returns both the default attributes and the newly saved ones:

HTTP/1.1 200 OK
Content-Type: application/json
[
    {
        "attributeId": "city",
        "type": "string",
        "indexed": "true"
    },
    {
        "attributeId": "state",
        "type": "category",
        "indexed": "true"
    },
    {
        "attributeId": "isTeacher",
        "type": "boolean",
        "indexed": "true"
    },
    {
        "attributeId": "zipCode",
        "type": "int",
        "indexed": "false"
    }
]

Add a New User

It is time to add a new user to the database. Let us consider the following donor:

FieldValue
userId5f24f7ece308e11c9e31a6b9ad53cf68
cityHempstead
stateNew York
isTeacherYes
zipCode115

We can add them to the database, by using the following command:

POST /v2/recomm/users HTTP/1.1

{
    "userId": "5f24f7ece308e11c9e31a6b9ad53cf68",
    "city": "Hempstead",
	"state": "New York",
	"isTeacher": "Yes",
	"zipCode": 115
}

The response is to this is:

HTTP/1.1 201 Created

Users can be created this way, or by means of an endpoint for massive data loading from a file. Go to Tutorial 2 for a detailed example about this option.

We can confirm that the donor has been correctly registered, using the following GET command:

GET /v2/recomm/users/5f24f7ece308e11c9e31a6b9ad53cf68 HTTP/1.1

In the response, we obtain the same data that was saved before:

HTTP/1.1 200 OK
Content-Type: application/json
{
    "city": "Hempstead",
    "state": "New York",
    "isTeacher": true,
    "zipCode": 115
}

Register Interactions

The basis of the recommender systems that the API provides is the set of interactions between users and items. In this section, we consider donations made by donors to the existing projects.

Create a Type of Interaction

First of all, the interaction itself needs to be defined. There are five parameters to customize it: weight, cumulative, interactionType, maxValue, and minValue (see the documentation for specific details about each of them). In this case, donations can be cumulative with a continuous nonnegative value. Thus, the command to create it is the following:

POST /v2/recomm/projects/interactions HTTP/1.1

{
    "interactionId": "donate",
    "type": "continuous",
    "cumulative": true,
    "weight": 1.0,
    "minValue": 0.0
}

The response to this is:

HTTP/1.1 201 Created

Note: We arbitrarily assigned weight 1.0 to the interaction, since we only consider one interaction type. This parameter becomes relevant when two or more interaction types come into play.

A GET command can confirm that the insertion is correct:

GET /v2/recomm/projects/interactions HTTP/1.1

The response contains all the parameters, including those that we did not fix:

HTTP/1.1 200 OK
Content-Type: application/json
[
    {
        "interactionId": "donate",
        "weight": 1.0,
        "cumulative": true,
        "interactionType": "continuous",
        "maxValue": "Infinity",
        "minValue": 0.0
    }
]

Register a New Interaction

Once the interaction is created, we can upload the donations to the projects. For example, let us consider the following:

FieldValue
userId5f24f7ece308e11c9e31a6b9ad53cf68
itemId69da11b15b82cf59c389ba81c444731e
interactionIddonate
timestamp2018-04-15 13:06:01
value25

Note: When inserting new interactions, timestamps need to be expressed as epoch times in milliseconds.

We can add it to the database by using the following command:

POST /v2/recomm/projects/interactions/donate/5f24f7ece308e11c9e31a6b9ad53cf68/69da11b15b82cf59c389ba81c444731e HTTP/1.1

{
    "timestamp": 1523797561000,
    "value": 25
}

As in previous cases, the response to this is:

HTTP/1.1 201 Created

To check that the previous command saved the data correctly, we can list the interactions made by that user:

GET /v2/recomm/projects/users/5f24f7ece308e11c9e31a6b9ad53cf68/donate HTTP/1.1

This command returns the unique interaction that the user has:

HTTP/1.1 200 OK
Content-Type: application/json
[
    {
        "itemId": "69da11b15b82cf59c389ba81c444731e",
        "timestamp": 1523797561000,
        "value": 25.0
    }
]

It is also possible to check by listing the donations made to the project:

GET /v2/recomm/projects/items/69da11b15b82cf59c389ba81c444731e/donate HTTP/1.1

In this case, the response is the following:

HTTP/1.1 200 OK
Content-Type: application/json
[
    {
        "userId": "5f24f7ece308e11c9e31a6b9ad53cf68",
        "timestamp": 1523797561000,
        "value": 25.0
    }
]