Skip to main content

On-Call Playbook

There are several common errors and issues that can occur when interacting with the price simulator. This playbook provides guidance on how to identify, troubleshoot, and resolve these issues. It includes a list of common errors, their potential causes, and mitigation steps to address them.

Playbook for DaaS Errors​

In practice, the vast majority of errors being surfaced from the price simulator are going to be due to its dependency on the data layer. If there is a data error, please contact the data team for further investigation, as there is unlikely anything that can be done on the client side except for bypassing the data layer by manually inserting historical data into the request payload.

HTTP 500 Error / Data Source Failure​

Note: This is likely going to be the issue with the price simulator the majority of the time that a page goes out for this service. The data team will need to be involved to fix the core issue, but there are mitigations that can be taken in the request payload to the price simulator, as discussed in greater detail in the mitigation steps below.

  • Description: This error occurs when the price simulator encounters an unhandled data error. The data team would need to fix the core issue, but there are mitigations that can be done on the client side in the interim.

  • Cause: This error is caused by an unhandled error in the data layer, such as a missing data source, incorrect data format, the service being down, or some other data-related issue.

  • Mitigation Steps:

    1. If the data source itself is not working, try another data feed by changing the data_feed key in the payload. The currently supported data feed options are biglake, biglake_trades, iceberg, parquet, and binance. Note that the asset pair may need to change based on the data feed being used.
    2. If no data feed is working as expected or changing the data feed is not viable, then historical data will need to manually be provided using the historical_data key in the payload. This will bypass the data layer and use the provided data directly. See the documentation for the expected format of the historical_data key. An example of the payload with historical data is as follows: {"assets": [{"symbol": ["ETH", "USDT"], "historical_data": {"Date": {"0": 1669427999999, "1": 1669431599999, "2": 1669435199999, "3": 1669438799999, "4": 1669442399999, "5": 1669445999999}, "Close": {"0": 1219.2, "1": 1218.22, "2": 1215.71, "3": 1222.97, "4": 1220.58, "5": 1216.9}}}, {"symbol": ["LINK", "USDT"], "historical_data": {"Date": {"0": 1669859999999, "1": 1669863599999, "2": 1669867199999, "3": 1669870799999, "4": 1669874399999, "5": 1669877999999}, "Close": {"0": 7.565, "1": 7.58, "2": 7.58, "3": 7.547, "4": 7.535, "5": 7.556}}}, {"symbol": ["USDT", "DAI"], "historical_data": {"Date": {"0": 1669859999999, "1": 1669863599999, "2": 1669867199999, "3": 1669870799999, "4": 1669874399999, "5": 1669877999999}, "Close": {"0": 1.0003, "1": 1.0004, "2": 1.0004, "3": 1.0004, "4": 1.0003, "5": 1.0004}}}]}
    3. If using historical data, this helps define the volatility shape, but may or may not be the most recent price data for each asset. If you opt to keep older historical data for the purposes of modeling volatility, but want to use a custom start time, then you must also define start_price in the asset-specific payload. This will be defined in each asset at the same hierarchical level as historical_data. See the documentation on Price Simulator Settings for further details.
    4. If this does not work or is not viable, then please reach out directly to the data team for further investigation. The points of contact for the data team are listed at the bottom of this playbook.

Web3 Connection to Ethereum Mainnet Fails​

Playbook for HTTP 400 Errors​

If an HTTP 400 error is being returned from the price simulator and causing an issue downstream, the following steps can be taken to troubleshoot and resolve the issue. Note that there may be multiple causes of an HTTP 400 error, so the first step is to investigate the message printed in the error and then proceed with the appropriate mitigation steps outlined below.

Error: All assets need to use historical_data payload or all need to not use historical_data​

  • Description: This error occurs when there is an inconsistency in the use of historical_data across the assets provided in the payload. The service expects either all assets to include historical_data or none of them to include it.

  • Cause: This error is caused when some assets in the payload have historical_data while others do not.

  • Mitigation Steps:

    1. Review the payload sent to the price simulator.
    2. Ensure consistency in the assets array: either all assets should have the historical_data key, or none should include it.
    3. Resend the request with the corrected payload.

Error: The price symbol must have two tokens in a list to be correctly defined​

  • Description: This error occurs when the symbol for any asset in the payload is not a list containing exactly two tokens.

  • Cause: This error is caused by an incorrectly formatted symbol in one or more assets, where the symbol is not a two-element list.

  • Mitigation Steps:

    1. Review the payload, focusing on the symbol key of each asset.
    2. Correct any symbol that is not a list of exactly two elements.
    3. Ensure that each symbol list contains valid token identifiers.
    4. Resend the request with the corrected payload.

Error: No price data found for the following assets: [list of assets]​

  • Description: This error indicates that the service could not find price data for one or more of the specified assets.

  • Cause: This error can be caused by various issues, such as incorrect asset symbols, missing data for the specified time range, or issues with the data source.

  • Mitigation Steps:

    1. Verify the symbols of the assets mentioned in the error message.
    2. Check the time range specified in the payload (start_block and end_block) to ensure it is correct and within a range that has data available.
    3. Confirm that the data source specified in the payload (data_feed) is correct and available.
    4. If using historical data, ensure the provided data is correctly formatted and contains entries for the specified assets.
    5. Resend the request after addressing the potential issues. If there are still issues, consult the above playbook entry for general DaaS errors and reach out to the data team.

General Mitigation Steps:​

  • Validate Payload Before Sending: Ensure that the payload adheres to the expected structure and data types before sending it to the price simulator. This can prevent many common errors.
  • Logging and Monitoring: Implement detailed logging on the client side to capture the payloads sent and the responses received. Monitoring these logs can help quickly identify and address issues.
  • Contact Support/Development Team: If errors persist or the cause is unclear, contact the support or development team with details of the payload and the errors encountered for further investigation.

Points of Contact​

If you encounter an issue with the price simulator that is not covered in this playbook or requires further investigation, please reach out to the following individuals for assistance:

  • Simulation Team: For issues related to the price simulator service, API, or client-side errors.
    1. Dan Shea - daniel@almanak.co
    2. Koen van Marrewijk - koen@almanak.co
  • Data Team: For issues related to data availability, data quality, or data layer errors.
    1. Shahril Kamaruzzaman - shahril@almanak.co
    2. Nasrudin bin Salim - nas@almanak.co