Classification

This page details the endpoint for sending images for classification through Foyer Insight. An SDK is provided at https://www.npmjs.com/package/@foyer-inc/insight-sdk

There is a 10MB limit on images sent to the classify endpoint.

Classify a single image

POST https://api.foyer.ai/Media/classify

This endpoint runs your image through classification and object detection. Accepts a single image with the parameters below, or an array of images.

Headers

Name

Type

Description

Authorization

string

Authorization token.

Request Body

Name

Type

Description

file

string

An image encoded as a base64 string

files

array

An array of images encoded as base64 strings

url

string

A URL to an image

urls

array

An array of URLs to images

force

boolean

A boolean representing whether to force a reclassification of an image

includeSegmentations

boolean

A boolean representing whether to return full segmentation masks for detections

includeTagpoints

string

A boolean representing whether to return a tagpoint attribute for each detection

detectionsRequested

array

An array of detection names as strings to be returned from the endpoint

{ 
    "classifications": [
        { 
            "name": "living_room", 
            "confidence": 0.9996274709701538, 
            "rank": 1 
        },
        { 
            "name": "outdoor", 
            "confidence": 0.7036548946876468, 
            "rank": 2 
        }
    ],
    "detections": [
        { 
            "class": "floor",
            "area":  0.1321029663085938, 
            "boundingBox": [
                            0,
                            0.091796875,
                            0.998046875,
                            0.896484375
                            ],
            "confidence": 0.4213932454586029,
            "attributes": [
                {
                "name": "floor_type",
                "value": "hardwood",
                "confidence": 0.923857
                }
            ],
            "segmentation": {'size': [512, 512], 'counts': 'YWY62[`1OdoN0ZXd1'}
        }
    ],
    "metadata": {
        "md5": "6897fa4c4cb0cc682ac75d90f83eafb5",
        "width": 1937,
        "height": 2583
    }
}

Only one of file, files, url or urls may be specified.

Example request body

{
    "url": "https://upload.wikimedia.org/wikipedia/commons/6/6d/Down_House.jpg",
    "force": true
}

Possible classifications

aerial_view

indoor

outdoor

bathroom

bedroom

community

gym

diagram

dining_room

front_of_structure

garage

kitchen

laundry

living_room

office

other

pool

closet

Possible object detections

fireplace

stairway

floor

ceiling

window

wall

stairs

cabinet

sink

counter

closet

toilet

kitchen_island

bar

countertop

bannister

booth

chandelier

shelf

stove

railing

refrigerator

column

bathtub

blind

escalator

door

fountain

stage

buffet_counter

conveyor

canopy

washing_machine

swimming_pool

oven

step

tank

microwave

dishwasher

hood

sconce

shower

radiator

Attributes

Attributes are more specific features gathered from the objects detected in the images. For the segmentations that have them, the attributes will give more robust information about the image. Attributes will always have a name field and a value field. Currently, the possible attributes are:

tagpoint: the tagpoint attribute's value is an array in the shape of [x, y] where x and y are percentage-based. They form a point that is guaranteed to be on the segmentation for tagging/display purposes.
is_stainless: A boolean. This attribute appears on refrigerator, stove, oven, washing_machine, hood, and dishwasher segmentations.
floor_type: A string of one of the possible flooring classifications (defined below). An additional float field, confidence, exists on this attribute.

Possible flooring classifications

A floor segmentation will automatically have a flooring type within its attributes.

carpet

stone

marble

hardwood

tile

other

Classify images from a RESO object

POST https://api.foyer.ai/Media/classifyReso

This endpoint runs multiple images through classification and object detection from a valid RESO payload.

Headers

Name

Type

Description

Authorization

string

Authorization token.

Request Body

Name

Type

Description

value

array

A single RESO object or an array of multiple. Objects must contain a Media Resource as defined by the RESO standard.

{
  "@odata.context": "http://odata.listhub.moveaws.com/odata/$metadata#Property",
  "value": [
    {
      "@odata.id": "http://odata.listhub.moveaws.com/odata/Property('3yd-A2SELL-281884')",
      "ListingKey": "3yd-A2SELL-281884",
      "ListingId": "281884",
      "PropertyType": "Land",
      "PropertySubType": "Other",
      "ListPrice": 21900,
      ...
      "Media": [
        {
          "MediaCategory": "Photo",
          "MediaURL": "http://photos.listhub.net/A2SELL/281884/1",
          "MediaKey": "2483763c0011d2a36483fb3f28ca8c00564abb4d",
          "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
          "ImageOf": [
            "Diagram"
          ]
        },
        {
          "MediaCategory": "Photo",
          "MediaURL": "http://photos.listhub.net/A2SELL/281884/2",
          "MediaKey": "192607ff01a04478df871a6146b5e44e9683deb2",
          "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
          "ImageOf": [
            "Outdoor"
          ]
        },
        {
          "MediaCategory": "Photo",
          "MediaURL": "http://photos.listhub.net/A2SELL/281884/3",
          "MediaKey": "748444298599641bba9ae7278f777759779afe9f",
          "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
          "ImageOf": [
            "Outdoor"
          ]
        },
        {
          "MediaCategory": "Photo",
          "MediaURL": "http://photos.listhub.net/A2SELL/281884/4",
          "MediaKey": "a763fdf6f243a9ade93a3695af5eaf2c7233829a",
          "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
          "ImageOf": [
            "Outdoor"
          ]
        },
        {
          "MediaCategory": "Photo",
          "MediaURL": "http://photos.listhub.net/A2SELL/281884/5",
          "MediaKey": "7d2531b1ddc50deb7c256c2c67c88a399a5b9b68",
          "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
          "ImageOf": [
            "Outdoor"
          ]
        },
        {
          "MediaCategory": "Photo",
          "MediaURL": "http://photos.listhub.net/A2SELL/281884/6",
          "MediaKey": "19683d3d369aaade3551c0c192b5c186de2c563b",
          "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
          "ImageOf": [
            "Outdoor"
          ]
        },
        {
          "MediaCategory": "Photo",
          "MediaURL": "http://photos.listhub.net/A2SELL/281884/7",
          "MediaKey": "4652e1f0e0a5400747b4f5f8f1eaa74d65edf943",
          "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
          "ImageOf": [
            "Outdoor"
          ]
        },
        {
          "MediaCategory": "Photo",
          "MediaURL": "http://photos.listhub.net/A2SELL/281884/8",
          "MediaKey": "cd4a82af4d64d5549031b1cbfbeee24a0103c00a",
          "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
          "ImageOf": [
            "Outdoor"
          ]
        }
      ]
    }
  ],
  "@odata.nextLink": "https://odata.listhub.moveaws.com/odata/Property?%24top=1&%24skiptoken=3yd-A2SELL-281884"
}

This function accepts a RESO payload and returns inference data from all provided Media objects. This returns the RESO data with additional classification and segmentation data added to it. To fit RESO guidelines where possible, output classifications are contained in the ImageOf field.

The classifyRESO endpoint does not return mask data. Mask data for images on a full listing can make the resulting payload many times larger. We can provide bulk inference with mask data included, but logistics for the large payloads must first be arranged.

See the RESO definition for a Media Resource. We require the MediaURL field to be defined for image classification.

Example request body

{
    "@odata.id": "http://odata.listhub.moveaws.com/odata/Property('3yd-A2SELL-281884')",
    "ListingKey": "3yd-A2SELL-281884",
    "ListingId": "281884",
    "PropertyType": "Land",
    "PropertySubType": "Other",
    "ListPrice": 21900,
    "Media": [
    {
        "MediaCategory": "Photo",
        "MediaURL": "http://photos.listhub.net/A2SELL/281884/1",
        "MediaKey": "2483763c0011d2a36483fb3f28ca8c00564abb4d",
        "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z"
    }
    ]
}

Example request return

{
    "@odata.id": "http://odata.listhub.moveaws.com/odata/Property('3yd-A2SELL-281884')",
    "ListingKey": "3yd-A2SELL-281884",
    "ListingId": "281884",
    "PropertyType": "Land",
    "PropertySubType": "Other",
    "ListPrice": 21900,
    "Media": [
        {
            "MediaCategory": "Photo",
            "MediaURL": "http://photos.listhub.net/A2SELL/281884/1",
            "MediaKey": "2483763c0011d2a36483fb3f28ca8c00564abb4d",
            "MediaModificationTimestamp": "2019-10-03T09:14:00.000Z",
            "ImageOf": [
                "diagram",
                "plat_map",
                "outdoor"
            ],
            "Segmap": [
                {
                    "class": "wall",
                    "attributes": [],
                    "area": 0.00493621826171875,
                    "boundingBox": [
                        0.408203125,
                        0.169921875,
                        0.974609375,
                        0.490234375
                    ],
                    "confidence": 0.4345703125
                },
                {
                    "class": "sky",
                    "attributes": [],
                    "area": 0.9121208190917969,
                    "boundingBox": [
                        0,
                        0,
                        0.998046875,
                        0.998046875
                    ],
                    "confidence": 0.93798828125
                },
                {
                    "class": "fence",
                    "attributes": [],
                    "area": 0.0113372802734375,
                    "boundingBox": [
                        0.546875,
                        0.248046875,
                        0.96875,
                        0.37890625
                    ],
                    "confidence": 0.47607421875
                }
            ]
        }
    ]
}

PreviousAuthorization NextValidation

Last updated 3 years ago