WHERE DO I START?
You are here:... (Miscellaneous) > Appendix > Custom Ingest > Custom Ingest FAQ/Examples

Custom Ingest FAQ and Examples

There are a few instances in which an error code is received:

  • The timestamp is not properly formatted. See the FAQ below for the proper format.
  • The timestamp is properly formatted; however, it is not within a period of time that will be processed by Netuitive.
  • Metrics or Samples have been sent without wrapping them in a valid Element consisting of an Element.id, Metric.id, and a Sample.metricId.

Frequently Asked Questions

  • What is the endpoint to get to Netuitive custom ingest?
    • Production: https://api.app.netuitive.com/ingest/{API_KEY}
    Note   Netuitive uses a POST method.
  • What is an API Key, and how do I get one/find mine?
    • An API Key is generated by Netuitive for your account. You can retrieve any of your API Keys from the API Keys page under the Account Profile drop-down menu.
  • What format do you require your timestamp to be in when sending samples?
    • Timestamps must be the number of milliseconds since the Unix Epoch. Now the stand date format on many Linux/Unix systems use seconds since the epoch and therefore require padding to be used. The following examples define ways to calculate the correct time in several circumstances:
      • Unix/Linux/Mac OS X using the date command returns seconds, so additional padding is needed to get it to milliseconds.
        • input: date +%s000
        • output: 144147539000
      • Java/Groovy contain the "System.currentTimeMillis()" method, which returns a properly formatted timestamp.
        • input: System.currentTimeMillis()
        • output: 1441475930587
  • What are valid timestamps accepted by Netuitive custom ingest endpoint?
    • The following formats have been tested successfully:
      • Unix epoch in milliseconds
        • 1441475739000
      • ISO 8601 formats:
        • 2015-09-07T14:26:47.457-0400
        • 2015-09-07T14:26:47-0400
        • 2015-09-08T13:39:33.948Z
    • The following ISO 8601 formats do not work:
      • 2015-09-07T14:26:47EDT
  • Can I send more than one Element at a time?
    • Yes, the posted JSON message is always an array of Elements. See the sample payload below. It sends two Elements at a time to custom ingest.
  • What is the metric name field used for?
    • Currently it is only used internally. In the future, we plan to use it in the UI similar to the Element name. Currently, the metric ID is also used for its name in the UI.
  • What is the element name?
    • The element name is used as the display name in the UI.
  • What client tools can I use to test sending data to Netuitive's ingest endpoint?
    • Command Line tools:
      • curl is the de facto command line tool that is available across most OSs. It is available on Linux/Unix/Mac OSs and available as a download on other OSs--including Windows--from the curl website. Examples below will use curl.
    • OS Agnostic UI tools:
  • What options can I send to the Netuitive's ingest endpoint?
  • What happens if the API detects unknown properties in a payload?
    • The API ignores unknown properties and allows the rest of the message to proceed if it is valid.
  • Is it possible to create relationships between elements in Netuitive's ingest endpoint?
    • Yes. Relationships can be added as an array of relations. Relationships can be uni- or bi-directional. Netuitive only creates the side of the relationship specified in the JSON payload. For more information, see our Ingest page.
      Example(s)   If you submit an element with "id" = "webserver01" that contained a relationship "fqn":"load-balancer-east", Netuitive would create a relationship from webserver01 to load-balancer-east. It would not automatically create a relationship from load-balander-east to webserver01; however, you can add the inverse relationship to load-balancer-east via a separate API call.

Example Scenario

Say you want to send 2 Elements to the Netuitive ingest endpoint: one server named "webhost01" and an instance of Tomcat 7 named "webapp01". For webhost01, you'd like to collect the memory utilization and cpu utilization, and you'd also like to associate an attribute noting what datacenter the server is runnign in. For webapp01, you'd like to collect requests per second and connection pool count, and you'd also like to associate an attribute that describes what webserver type the instance is running.

To implement the plan outlined above:

  1. Array of Elements (JSON root)
    1. id: Choose a unique (across your environment) ID for each of your elements.
    2. name: The name will be displayed in the Netuitive UI.
    3. type: This should describe the type of asset you are using (e.g., server, firewall, database, webserver, storage).
  2. Metrics: define the metrics you want to capture for each element
    1. id: The ID of the metric. This should uniquely identify it within the Element. Netuitive also uses this as the display name in the UI.
    2. name: Currently the metric name is not used as the ID is used for the display name in the UI.
  3. Samples:
    1. metricId: This relates the sample to the metric.id.
    2. timestamp: UTC time in milliseconds since the epoch.
    3. val: The value for this sample.

IngestRunner

The ingestRunner.sh shell script is a simple tool that helps submit custom messages to the Netuitive custom endpoint. It allows you to create a JSON file (as shown in the example below) and replace all of the timestamps with the literal TIMESTAMP. When you run the shell script, it will replace all occurences of TIMESTAMP with the current, valid timestamp.