Anomaly Detector v1.1-preview.2

The Anomaly Detector service detects anomalies and trace root causes automatically in time series data. It supports several functionalities:

  • Detecting anomalies in single time-series, including entire, last and change point detection. With this univariate anomaly detection capability, business customers can discover incidents and establish a logic flow for root cause analysis.
  • Detecting anomalies in multiple time-series. This set of multivariate anomaly detection APIs in Anomaly Detector analyze dependencies and inter-correlation changes between different signals. It enables customers to gather a group of related time-series and detect failures in a system level.
  • Indicate root causes that contribute the most on an aggregated anomaly point. Accurate dimension can be drilled down for both one measure and multiple measures. Meaningful interpretation is also provided.

To ensure online service quality is one of the main reasons we developed this service. Our team is dedicated to continuing to improve the Anomaly Detector service to provide precise results.

This Univariate Anomaly Detection is currently available in:

This Multivariate Anomaly Detection is currently available in:

  • East US - eastus.api.cognitive.microsoft.com
  • East US 2 - eastus2.api.cognitive.microsoft.com
  • South Central US - southcentralus.api.cognitive.microsoft.com
  • UK South - uksouth.api.cognitive.microsoft.com
  • West Europe - westeurope.api.cognitive.microsoft.com
  • West US 2 - westus2.api.cognitive.microsoft.com

The Root Cause Analysis is currently available in:

  • West US 2 - westus2.api.cognitive.microsoft.com
  • Central US EUAP - ameuse2.api.cognitive.microsoft.com

Multivariate Anomaly Detection - Train a Multivariate Anomaly Detection Model

Create and train a multivariate anomaly detection model. The request must include a source parameter to indicate an externally accessible Azure storage Uri (preferably a Shared Access Signature Uri). All time-series used in generate the model must be zipped into one single file. Each time-series will be in a single CSV file in which the first column is timestamp and the second column is value.

Select the testing console in the region where you created your resource:

Open API testing console

Request URL

Request headers

string
Media type of the body sent to the API.
string
Subscription key which provides access to this API. Found in your Cognitive Services accounts.

Request body

Training request

{
  "slidingWindow": 200,
  "alignPolicy": {
    "alignMode": "Outer",
    "fillNAMethod": "Linear",
    "paddingValue": 0
  },
  "source": "{your_sas_url}",
  "startTime": "2019-04-09T00:00:00+00:00",
  "endTime": "2019-04-21T00:00:00+00:00",
  "displayName": "SampleRequest"
}
{
  "description": "Train result of a model including status, errors and diagnose info for model and variables.",
  "required": [
    "endTime",
    "source",
    "startTime"
  ],
  "type": "object",
  "properties": {
    "slidingWindow": {
      "description": "An optional field, indicating how many previous points will be used to compute the anomaly score of the subsequent point.",
      "type": "integer",
      "example": 200
    },
    "alignPolicy": {
      "type": "object",
      "properties": {
        "alignMode": {
          "description": "An optional field, indicating how we align different variables to the same time-range. Either Inner or Outer. ",
          "enum": [
            "Inner",
            "Outer"
          ],
          "type": "string",
          "example": "Outer",
          "x-ms-enum": {
            "name": "alignMode",
            "modelAsString": false
          }
        },
        "fillNAMethod": {
          "description": "An optional field, indicating how missing values will be filled. One of Previous, Subsequent, Linear, Zero, Fixed, and NotFill. Cannot be set to NotFill, when the alignMode is Outer.",
          "enum": [
            "Previous",
            "Subsequent",
            "Linear",
            "Zero",
            "Fixed",
            "NotFill"
          ],
          "type": "string",
          "example": "Linear",
          "x-ms-enum": {
            "name": "fillNAMethod",
            "modelAsString": true
          }
        },
        "paddingValue": {
          "description": "An optional field. Required when fillNAMethod is Fixed.",
          "type": "number"
        }
      }
    },
    "source": {
      "description": "Source link to the input variables. Each variable should be a csv file with two columns, `timestamp` and `value`. By default, the file name of the variable will be used as its variable name.",
      "type": "string",
      "example": "{your_sas_url}"
    },
    "startTime": {
      "format": "date-time",
      "description": "A required field, indicating the start time of training data. Should be date-time.",
      "type": "string",
      "example": "2019-04-01T00:00:00Z"
    },
    "endTime": {
      "format": "date-time",
      "description": "A required field, indicating the end time of training data. Should be date-time.",
      "type": "string",
      "example": "2019-04-02T00:00:00Z"
    },
    "displayName": {
      "description": "An optional field. The name of the model whose maximum length is 24.",
      "maxLength": 24,
      "type": "string",
      "example": "DevOps-MultiAD"
    },
    "status": {
      "description": "Model training status.",
      "enum": [
        "CREATED",
        "RUNNING",
        "READY",
        "FAILED"
      ],
      "type": "string",
      "readOnly": true,
      "example": "READY",
      "x-ms-enum": {
        "name": "modelStatus",
        "modelAsString": false
      }
    },
    "errors": {
      "description": "Error messages when failed to create a model.",
      "type": "array",
      "items": {
        "required": [
          "code",
          "message"
        ],
        "type": "object",
        "properties": {
          "code": {
            "description": "The error code.",
            "type": "string"
          },
          "message": {
            "description": "The message explaining the error reported by the service.",
            "type": "string"
          }
        }
      },
      "readOnly": true
    },
    "diagnosticsInfo": {
      "type": "object",
      "properties": {
        "modelState": {
          "type": "object",
          "properties": {
            "epochIds": {
              "description": "Epoch id",
              "type": "array",
              "items": {
                "type": "integer"
              },
              "example": [
                10,
                20,
                30,
                40,
                50,
                60,
                70,
                80,
                90,
                100
              ]
            },
            "trainLosses": {
              "type": "array",
              "items": {
                "type": "number"
              },
              "example": [
                0.629132807254791,
                0.167132690548897,
                0.123542480170727,
                0.102596640586853,
                0.0958492755889893,
                0.0906995236873627,
                0.0868601649999619,
                0.0860302299261093,
                0.0828735455870628,
                0.0823553800582886
              ]
            },
            "validationLosses": {
              "type": "array",
              "items": {
                "type": "number"
              },
              "example": [
                0.957342743873596,
                0.318222999572754,
                0.340264648199081,
                0.276534885168076,
                0.257401436567307,
                0.269286036491394,
                0.26372304558754,
                0.259083300828934,
                0.259716689586639,
                0.259735018014908
              ]
            },
            "latenciesInSeconds": {
              "type": "array",
              "items": {
                "type": "number"
              },
              "example": [
                0.339859485626221,
                0.365966558456421,
                0.373606443405151,
                0.351340770721436,
                0.337030410766602,
                0.31876277923584,
                0.328330993652344,
                0.350358724594116,
                0.308002471923828,
                0.332794666290283
              ]
            }
          }
        },
        "variableStates": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "variable": {
                "description": "Variable name.",
                "type": "string"
              },
              "filledNARatio": {
                "description": "Proportion of NaN values filled of the variable.",
                "maximum": 1,
                "minimum": 0,
                "type": "number"
              },
              "effectiveCount": {
                "description": "Number of effective points counted.",
                "type": "integer"
              },
              "startTime": {
                "format": "date-time",
                "description": "Start time of the variable.",
                "type": "string"
              },
              "endTime": {
                "format": "date-time",
                "description": "End time of the variable.",
                "type": "string"
              }
            }
          },
          "example": [
            {
              "variable": "ad_input",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            },
            {
              "variable": "ad_on_timer_output",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            },
            {
              "variable": "ingestion",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            },
            {
              "variable": "data_in_speed",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            },
            {
              "variable": "cpu",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            },
            {
              "variable": "ad_series_init",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            },
            {
              "variable": "F-Link_last_check_point_duration",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            },
            {
              "variable": "data_out_speed",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            },
            {
              "variable": "ad_output",
              "filledNARatio": 0,
              "effectiveCount": 1441,
              "startTime": "2019-04-01T00:00:00Z",
              "endTime": "2019-04-02T00:00:00Z"
            }
          ]
        }
      },
      "readOnly": true
    }
  },
  "example": "{\r\n  \"slidingWindow\": 200,\r\n  \"alignPolicy\": {\r\n    \"alignMode\": \"Outer\",\r\n    \"fillNAMethod\": \"Linear\",\r\n    \"paddingValue\": 0\r\n  },\r\n  \"source\": \"{your_sas_url}\",\r\n  \"startTime\": \"2019-04-09T00:00:00+00:00\",\r\n  \"endTime\": \"2019-04-21T00:00:00+00:00\",\r\n  \"displayName\": \"SampleRequest\"\r\n}"
}

Response 201

Submit a multivariate model training task successfully.

Response 400

Possible Errors:

  • TooManyModel
    This subscription has reached the maximum number of models.
  • TooManyRunningModel
    This subscription has reached the maximum number of running models.
  • InvalidSlidingWindow
    The 'slidingWindow' field must be an integer between 28 and 2880.
  • BadArgument
    The 'source' field is required in the request.
    The 'startTime' field is required in the request.
    The 'endTime' field is required in the request.
    Invalid Timestamp format.
    The displayName length exceeds maximum allowed length 24."
  • VariableNotExist
    The corresponding file of the variable does not exist.
  • MergeDataFailed
    Data provided could be merged into a dataframe. Possibly due to wrong folder structure or data format, invalid column names.
    Folder structure may be changed after compression. Please check the structure is desired after extraction.
    Please refer to the sample data to prepare your own data.
  • ColumnNotFound
    Could not find column "timestamp" in the merged dataframe.
  • NumColumnsMismatch
    Number of columns of merged data does not match the number of variables.
  • CorruptedData
    Data provided could be processed. Possibly due to wrong folder structure or data format, invalid column names.
    Folder structure may be changed after compression. Please check the structure is desired after extraction.
    Please refer to the sample data to prepare your own data. 

{
  "code": "TooManyModel",
  "message": "This subscription has reached the maximum number of models."
}
{
  "required": [
    "code",
    "message"
  ],
  "type": "object",
  "properties": {
    "code": {
      "description": "The error code.",
      "type": "string"
    },
    "message": {
      "description": "The message explaining the error reported by the service.",
      "type": "string"
    }
  }
}

Response 403

The certificate you provided is not accepted by server.

Response 405

Method Not Allowed.

Response 500

Internal Server Error.

Code samples

@ECHO OFF

curl -v -X POST "https://*.cognitiveservices.azure.com/anomalydetector/v1.1-preview.2/multivariate/models"
-H "Content-Type: application/json"
-H "Ocp-Apim-Subscription-Key: {subscription key}"

--data-ascii "{body}" 

using System;
using System.Net.Http.Headers;
using System.Text;
using System.Net.Http;
using System.Web;

namespace CSHttpClientSample
{
    static class Program
    {
        static void Main()
        {
            MakeRequest();
            Console.WriteLine("Hit ENTER to exit...");
            Console.ReadLine();
        }
        
        static async void MakeRequest()
        {
            var client = new HttpClient();
            var queryString = HttpUtility.ParseQueryString(string.Empty);

            // Request headers
            client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "{subscription key}");

            var uri = "https://*.cognitiveservices.azure.com/anomalydetector/v1.1-preview.2/multivariate/models?" + queryString;

            HttpResponseMessage response;

            // Request body
            byte[] byteData = Encoding.UTF8.GetBytes("{body}");

            using (var content = new ByteArrayContent(byteData))
            {
               content.Headers.ContentType = new MediaTypeHeaderValue("< your content type, i.e. application/json >");
               response = await client.PostAsync(uri, content);
            }

        }
    }
}	
// // This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
import java.net.URI;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

public class JavaSample 
{
    public static void main(String[] args) 
    {
        HttpClient httpclient = HttpClients.createDefault();

        try
        {
            URIBuilder builder = new URIBuilder("https://*.cognitiveservices.azure.com/anomalydetector/v1.1-preview.2/multivariate/models");


            URI uri = builder.build();
            HttpPost request = new HttpPost(uri);
            request.setHeader("Content-Type", "application/json");
            request.setHeader("Ocp-Apim-Subscription-Key", "{subscription key}");


            // Request body
            StringEntity reqEntity = new StringEntity("{body}");
            request.setEntity(reqEntity);

            HttpResponse response = httpclient.execute(request);
            HttpEntity entity = response.getEntity();

            if (entity != null) 
            {
                System.out.println(EntityUtils.toString(entity));
            }
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
    }
}


<!DOCTYPE html>
<html>
<head>
    <title>JSSample</title>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
</head>
<body>

<script type="text/javascript">
    $(function() {
        var params = {
            // Request parameters
        };
      
        $.ajax({
            url: "https://*.cognitiveservices.azure.com/anomalydetector/v1.1-preview.2/multivariate/models?" + $.param(params),
            beforeSend: function(xhrObj){
                // Request headers
                xhrObj.setRequestHeader("Content-Type","application/json");
                xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key","{subscription key}");
            },
            type: "POST",
            // Request body
            data: "{body}",
        })
        .done(function(data) {
            alert("success");
        })
        .fail(function() {
            alert("error");
        });
    });
</script>
</body>
</html>

#import <Foundation/Foundation.h>

int main(int argc, const char * argv[])
{
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    
    NSString* path = @"https://*.cognitiveservices.azure.com/anomalydetector/v1.1-preview.2/multivariate/models";
    NSArray* array = @[
                         // Request parameters
                         @"entities=true",
                      ];
    
    NSString* string = [array componentsJoinedByString:@"&"];
    path = [path stringByAppendingFormat:@"?%@", string];

    NSLog(@"%@", path);

    NSMutableURLRequest* _request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:path]];
    [_request setHTTPMethod:@"POST"];
    // Request headers
    [_request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
    [_request setValue:@"{subscription key}" forHTTPHeaderField:@"Ocp-Apim-Subscription-Key"];
    // Request body
    [_request setHTTPBody:[@"{body}" dataUsingEncoding:NSUTF8StringEncoding]];
    
    NSURLResponse *response = nil;
    NSError *error = nil;
    NSData* _connectionData = [NSURLConnection sendSynchronousRequest:_request returningResponse:&response error:&error];

    if (nil != error)
    {
        NSLog(@"Error: %@", error);
    }
    else
    {
        NSError* error = nil;
        NSMutableDictionary* json = nil;
        NSString* dataString = [[NSString alloc] initWithData:_connectionData encoding:NSUTF8StringEncoding];
        NSLog(@"%@", dataString);
        
        if (nil != _connectionData)
        {
            json = [NSJSONSerialization JSONObjectWithData:_connectionData options:NSJSONReadingMutableContainers error:&error];
        }
        
        if (error || !json)
        {
            NSLog(@"Could not parse loaded json with error:%@", error);
        }
        
        NSLog(@"%@", json);
        _connectionData = nil;
    }
    
    [pool drain];

    return 0;
}

<?php
// This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
require_once 'HTTP/Request2.php';

$request = new Http_Request2('https://*.cognitiveservices.azure.com/anomalydetector/v1.1-preview.2/multivariate/models');
$url = $request->getUrl();

$headers = array(
    // Request headers
    'Content-Type' => 'application/json',
    'Ocp-Apim-Subscription-Key' => '{subscription key}',
);

$request->setHeader($headers);

$parameters = array(
    // Request parameters
);

$url->setQueryVariables($parameters);

$request->setMethod(HTTP_Request2::METHOD_POST);

// Request body
$request->setBody("{body}");

try
{
    $response = $request->send();
    echo $response->getBody();
}
catch (HttpException $ex)
{
    echo $ex;
}

?>
########### Python 2.7 #############
import httplib, urllib, base64

headers = {
    # Request headers
    'Content-Type': 'application/json',
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.urlencode({
})

try:
    conn = httplib.HTTPSConnection('*.cognitiveservices.azure.com')
    conn.request("POST", "/anomalydetector/v1.1-preview.2/multivariate/models?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################

########### Python 3.2 #############
import http.client, urllib.request, urllib.parse, urllib.error, base64

headers = {
    # Request headers
    'Content-Type': 'application/json',
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.parse.urlencode({
})

try:
    conn = http.client.HTTPSConnection('*.cognitiveservices.azure.com')
    conn.request("POST", "/anomalydetector/v1.1-preview.2/multivariate/models?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################
require 'net/http'

uri = URI('https://*.cognitiveservices.azure.com/anomalydetector/v1.1-preview.2/multivariate/models')
uri.query = URI.encode_www_form({
})

request = Net::HTTP::Post.new(uri.request_uri)
# Request headers
request['Content-Type'] = 'application/json'
# Request headers
request['Ocp-Apim-Subscription-Key'] = '{subscription key}'
# Request body
request.body = "{body}"

response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
    http.request(request)
end

puts response.body