Elasticsearch data structure

This is the data structure of the Elasticsearch response for each call. We use it to design our dashboard charts. Fields are in use right now: request_at, request_url, request_ip, request_ip_country, response_time.

Overview

field	type	example	notes
api_key	string	8Uq3GR9sHiTrazBlEnNyGAfCNlPXULakC6s12ka2
user_id	string
request_accept	string	application/json
request_at	date	1475148774275
request_basic_auth_username	string
request_connection	string	close
request_content_type	string	application/json
request_host	string	nightly.apinf.io:3002
request_ip	string	87.94.164.76
request_ip_country	string	FI
request_ip_region	string	15
request_ip_city	string	Tampere
request_path	string	/api-umbrella/v1/apis/
request_path_hierarchy	array(string)	["0/nightly.apinf.io:3002/", "1/nightly.apinf.io:3002/api-umbrella/", "2/nightly.apinf.io:3002/api-umbrella/v1/", "3/nightly.apinf.io:3002/api-umbrella/v1/apis"]
request_query	string
request_referer
request_scheme	string	https
request_size	integer	590
response_cache	string	MISS
response_content_encoding
response_status	short
response_age	integer
response_content_length	integer	1436
response_content_type	string	application/json; charset=utf-8
response_size	integer	2280
response_server	string	openresty
response_status	string	201
response_time	integer	149
request_method	string	POST
backend_response_time	integer	131
proxy_overhead	integer	3
user_email	string	nightly@apinf.io
user_id	string	844433ba-5dfd-4734-af0d-09bd40636660
user_registration_source	string	web
internal_response_time	float	??	May no longer be in use
internal_gatekeeper_time	float	0
request_ip_location	geo_point	{ lat: 61.5, lon: 23.75 }

Raw example

{ _index: 'api-umbrella-logs-v1-2017-03',
  _type: 'log',
  _id: 'aqq7k887q9339spvrih0',
  _score: null,
  _source:
    { api_key: '[actual user's API key]',
    backend_response_time: 161,
    gatekeeper_denied_code: null,
    internal_gatekeeper_time: 0,
    proxy_overhead: 7,
    request_accept: '*/*',
    request_accept_encoding: 'gzip, deflate, sdch, br',
    request_at: 1489580360900,
    request_basic_auth_username: null,
    request_connection: 'keep-alive',
    request_content_type: null,
    request_hierarchy: [Object],
    request_host: 'nightly.apinf.io:3002',
    request_ip: '109.204.245.84',
    request_ip_city: 'Tampere',
    request_ip_country: 'FI',
    request_ip_location: [Object],
    request_ip_region: '15',
    request_method: 'GET',
    request_origin: null,
    request_path: '/api-umbrella/v1/analytics/drilldown.json',
    request_query: [Object],
    request_referer: 'https://nightly.apinf.io:3002/admin/',
    request_scheme: 'https',
    request_size: 1326,
    request_url: 'https://nightly.apinf.io:3002/api-umbrella/v1/analytics/drilldown.json?tz=Europe%2FHelsinki&search=&start_at=2017-02-14&end_at=2017-03-15&query=%7B%22condition%22%3A%22AND%22%2C%22rules%22%3A%5B%7B%22field%22%3A%22gatekeeper_denied_code%22%2C%22id%22%3A%22gatekeeper_denied_code%22%2C%22input%22%3A%22select%22%2C%22operator%22%3A%22is_null%22%2C%22type%22%3A%22string%22%2C%22value%22%3Anull%7D%5D%7D&interval=day&prefix=0%2F&beta_analytics=false',
    request_user_agent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
    request_user_agent_family: 'Chrome',
    request_user_agent_type: 'Browser',
    response_age: 0,
    response_cache: 'MISS',
    response_content_encoding: 'gzip',
    response_content_length: null,
    response_content_type: 'application/json; charset=utf-8',
    response_server: 'openresty',
    response_size: 1439,
    response_status: 200,
    response_time: 168,
    response_transfer_encoding: 'chunked',
    user_email: 'web.admin.ajax@internal.apiumbrella',
    user_id: '70a3c56f-5764-4a69-9b2a-c87aff573735',
    user_registration_source: 'seed' },
sort: [ 1489580360900 ]
}

Fields description:

• api_key: API key used to make the request
• backend_response_time:
• gatekeeper_denied_code:
• internal_gatekeeper_time:
• proxy_overhead:
• request_accept:
• request_accept_encoding:
• request_at: Timestamp of a request
• request_basic_auth_username:
• request_connection:
• request_content_type:
• request_hierarchy:
• request_host:
• request_ip: IP address
• request_ip_city: City
• request_ip_country: Country
• request_ip_location:
• request_ip_region:
• request_method: Method used to make a request. Can be POST, GET, PUT, PATCH, DELETE
• request_origin:
• request_path:
• request_query:
• request_referer:
• request_scheme:
• request_size:
• request_url:
• request_user_agent:
• request_user_agent_family:
• request_user_agent_type:
• response_age:
• response_cache:
• response_content_encoding:
• response_content_length:
• response_content_type:
• response_server:
• response_size:
• response_status: Response status code
• response_time: Response time
• response_transfer_encoding:
• user_email:
• user_id:
• user_registration_source:
• sort: