# Classification
{% hint style="warning" %}
There is a 10MB limit on images sent to the classify endpoint.
{% endhint %}
## Classify a single image
`POST` `https://api.foyer.ai/Media/classify`
This endpoint runs your image through classification and object detection. Accepts a single image with the parameters below, or an array of images.
#### Headers
| Name | Type | Description |
| ------------- | ------ | -------------------- |
| Authorization | string | Authorization token. |
#### Request Body
| Name | Type | Description |
| -------------------- | ------- | -------------------------------------------------------------------------------- |
| file | string | An image encoded as a base64 string |
| files | array | An array of images encoded as base64 strings |
| url | string | A URL to an image |
| urls | array | An array of URLs to images |
| force | boolean | A boolean representing whether to force a reclassification of an image |
| includeSegmentations | boolean | A boolean representing whether to return full segmentation masks for detections |
| includeTagpoints | string | A boolean representing whether to return a tagpoint attribute for each detection |
| detectionsRequested | array | An array of detection names as strings to be returned from the endpoint |
{% tabs %}
{% tab title="200 Image successfully classified.
All decimals are percents.
Indoor and outdoor classifications will always be ranked last, regardless of confidence." %}
```javascript
{
"classifications": [
{
"name": "living_room",
"confidence": 0.9996274709701538,
"rank": 1
},
{
"name": "outdoor",
"confidence": 0.7036548946876468,
"rank": 2
}
],
"detections": [
{
"class": "floor",
"area": 0.1321029663085938,
"boundingBox": [
0,
0.091796875,
0.998046875,
0.896484375
],
"confidence": 0.4213932454586029,
"attributes": [
{
"name": "floor_type",
"value": "hardwood",
"confidence": 0.923857
}
],
"segmentation": {'size': [512, 512], 'counts': 'YWY62[`1OdoN0ZXd1'}
}
],
"metadata": {
"md5": "6897fa4c4cb0cc682ac75d90f83eafb5",
"width": 1937,
"height": 2583
}
}
```
{% endtab %}
{% tab title="401 Invalid authentication token." %}
```javascript
{
"statusCode": 401,
"error": "Unauthorized",
"message": "Invalid credentials",
"attributes": {
"error": "Invalid credentials"
}
}
```
{% endtab %}
{% tab title="403 No associated account found." %}
```
```
{% endtab %}
{% endtabs %}
{% hint style="warning" %}
Only one of file, files, url or urls may be specified.
{% endhint %}
### Example request body
```javascript
{
"url": "https://upload.wikimedia.org/wikipedia/commons/6/6d/Down_House.jpg",
"force": true
}
```
### Possible classifications
| | | |
| :------------------: | :----------: | :----------: |
| aerial\_view | indoor | outdoor |
| bathroom | bedroom | community |
| gym | diagram | dining\_room |
| front\_of\_structure | garage | kitchen |
| laundry | living\_room | office |
| other | pool | closet |
### Possible object detections
{% tabs %}
{% tab title="Features" %}
| | | |
| :-------------: | :-------: | :--------------: |
| fireplace | stairway | floor |
| ceiling | window | wall |
| stairs | cabinet | sink |
| counter | closet | toilet |
| kitchen\_island | bar | countertop |
| bannister | booth | chandelier |
| shelf | stove | railing |
| refrigerator | column | bathtub |
| blind | escalator | door |
| fountain | stage | buffet\_counter |
| conveyor | canopy | washing\_machine |
| swimming\_pool | oven | step |
| tank | microwave | dishwasher |
| hood | sconce | shower |
| radiator | | |
| {% endtab %} | | |
{% tab title="Furniture" %}
| | | |
| :-----------: | :-----------: | :--------: |
| table | curtain | chair |
| bench | couch | chest |
| mirror | rug | armchair |
| seat | desk | stand |
| coffee\_table | pool\_table | grandstand |
| display\_case | pillow | bookcase |
| ottoman | swivel\_chair | stool |
| cradle | trashcan | bulletin |
| bed | | |
| {% endtab %} | | |
{% tab title="Items" %}
| | | |
| :----------: | :-------------: | :----------------: |
| poster | painting | lamp |
| computer | cushion | box |
| sign | screen | book |
| toy | arcade\_machine | towel |
| television | apparel | light |
| pole | bottle | basket |
| ball | barrel | bag |
| food | pot | projection\_screen |
| vase | sculpture | blanket |
| tray | fan | crt |
| plate | monitor | drinking\_glass |
| clock | flag | |
| {% endtab %} | | |
{% tab title="Outdoor" %}
| | | |
| :------------: | :---------: | :--------: |
| sky | tree | road |
| building | grass | sidewalk |
| ground | mountain | plant |
| car | water | house |
| sea | field | fence |
| stone | sand | skyscraper |
| path | runway | river |
| bridge | flower | hill |
| palm | boat | hut |
| truck | tower | bus |
| awning | streetlight | airplane |
| dirt\_track | land | van |
| tent | waterfall | ship |
| motorbike | bicycle | lake |
| traffic\_light | pier | |
| {% endtab %} | | |
| {% endtabs %} | | |
### Attributes
Attributes are more specific features gathered from the objects detected in the images. For the segmentations that have them, the attributes will give more robust information about the image. Attributes will always have a name field and a value field. Currently, the possible attributes are:
* `tagpoint`: the `tagpoint` attribute's value is an array in the shape of \[x, y] where x and y are percentage-based. They form a point that is guaranteed to be on the segmentation for tagging/display purposes.
* `is_stainless`: A boolean. This attribute appears on refrigerator, stove, oven, washing\_machine, hood, and dishwasher segmentations.
* `floor_type`: A string of one of the possible flooring classifications (defined below). An additional float field, `confidence`, exists on this attribute.
### Possible flooring classifications
A floor segmentation will automatically have a flooring type within its attributes.
| | | |
| :------: | :---: | :----: |
| carpet | stone | marble |
| hardwood | tile | other |
## Classify images from a RESO object
`POST` `https://api.foyer.ai/Media/classifyReso`
This endpoint runs multiple images through classification and object detection from a valid RESO payload.
#### Headers
| Name | Type | Description |
| ------------- | ------ | -------------------- |
| Authorization | string | Authorization token. |
#### Request Body
| Name | Type | Description |
| ----- | ----- | -------------------------------------------------------------------------------------------------------------------- |
| value | array | A single RESO object or an array of multiple. Objects must contain a Media Resource as defined by the RESO standard. |
{% tabs %}
{% tab title="200 Images successfully classified. Classifier information will be added to the ImageOf field of each Media Resource." %}
```javascript
{
"@odata.context": "http://odata.listhub.moveaws.com/odata/$metadata#Property",
"value": [
{
"@odata.id": "http://odata.listhub.moveaws.com/odata/Property('3yd-A2SELL-281884')",
"ListingKey": "3yd-A2SELL-281884",
"ListingId": "281884",
"PropertyType": "Land",
"PropertySubType": "Other",
"ListPrice": 21900,
...
"Media": [
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/1",
"MediaKey": "2483763c0011d2a36483fb3f28ca8c00564abb4d",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"Diagram"
]
},
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/2",
"MediaKey": "192607ff01a04478df871a6146b5e44e9683deb2",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"Outdoor"
]
},
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/3",
"MediaKey": "748444298599641bba9ae7278f777759779afe9f",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"Outdoor"
]
},
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/4",
"MediaKey": "a763fdf6f243a9ade93a3695af5eaf2c7233829a",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"Outdoor"
]
},
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/5",
"MediaKey": "7d2531b1ddc50deb7c256c2c67c88a399a5b9b68",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"Outdoor"
]
},
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/6",
"MediaKey": "19683d3d369aaade3551c0c192b5c186de2c563b",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"Outdoor"
]
},
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/7",
"MediaKey": "4652e1f0e0a5400747b4f5f8f1eaa74d65edf943",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"Outdoor"
]
},
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/8",
"MediaKey": "cd4a82af4d64d5549031b1cbfbeee24a0103c00a",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"Outdoor"
]
}
]
}
],
"@odata.nextLink": "https://odata.listhub.moveaws.com/odata/Property?%24top=1&%24skiptoken=3yd-A2SELL-281884"
}
```
{% endtab %}
{% tab title="401 Invalid authentication token." %}
```javascript
{
"statusCode": 401,
"error": "Unauthorized",
"message": "Invalid credentials",
"attributes": {
"error": "Invalid credentials"
}
}
```
{% endtab %}
{% tab title="403 No associated account found." %}
```
```
{% endtab %}
{% endtabs %}
This function accepts a RESO payload and returns inference data from all provided Media objects. This returns the RESO data with additional classification and segmentation data added to it. To fit RESO guidelines where possible, output classifications are contained in the `ImageOf` field.
{% hint style="warning" %}
The classifyRESO endpoint does not return mask data. Mask data for images on a full listing can make the resulting payload many times larger. We can provide bulk inference with mask data included, but logistics for the large payloads must first be arranged.
{% endhint %}
{% hint style="info" %}
See [the RESO definition for a Media Resource](https://ddwiki.reso.org/display/DDW17/Media+Resource). We require the `MediaURL` field to be defined for image classification.
{% endhint %}
### Example request body
```json
{
"@odata.id": "http://odata.listhub.moveaws.com/odata/Property('3yd-A2SELL-281884')",
"ListingKey": "3yd-A2SELL-281884",
"ListingId": "281884",
"PropertyType": "Land",
"PropertySubType": "Other",
"ListPrice": 21900,
"Media": [
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/1",
"MediaKey": "2483763c0011d2a36483fb3f28ca8c00564abb4d",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z"
}
]
}
```
### Example request return
```json
{
"@odata.id": "http://odata.listhub.moveaws.com/odata/Property('3yd-A2SELL-281884')",
"ListingKey": "3yd-A2SELL-281884",
"ListingId": "281884",
"PropertyType": "Land",
"PropertySubType": "Other",
"ListPrice": 21900,
"Media": [
{
"MediaCategory": "Photo",
"MediaURL": "http://photos.listhub.net/A2SELL/281884/1",
"MediaKey": "2483763c0011d2a36483fb3f28ca8c00564abb4d",
"MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
"ImageOf": [
"diagram",
"plat_map",
"outdoor"
],
"Segmap": [
{
"class": "wall",
"attributes": [],
"area": 0.00493621826171875,
"boundingBox": [
0.408203125,
0.169921875,
0.974609375,
0.490234375
],
"confidence": 0.4345703125
},
{
"class": "sky",
"attributes": [],
"area": 0.9121208190917969,
"boundingBox": [
0,
0,
0.998046875,
0.998046875
],
"confidence": 0.93798828125
},
{
"class": "fence",
"attributes": [],
"area": 0.0113372802734375,
"boundingBox": [
0.546875,
0.248046875,
0.96875,
0.37890625
],
"confidence": 0.47607421875
}
]
}
]
}
```