Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document capabilities with more clarity #2123

Open
italodamato opened this issue Dec 17, 2024 · 8 comments
Open

Document capabilities with more clarity #2123

italodamato opened this issue Dec 17, 2024 · 8 comments
Assignees
Labels
docs Relating to documentation

Comments

@italodamato
Copy link

Bug Report

Expected Behavior

BinanceSpotInstrumentProvider should be able to load all the instruments on init.

Actual Behavior

Failing with msgspec.DecodeError: JSON is malformed: invalid character (byte 0) when using Binance Margin. It works on testnet and SPOT.

Steps to Reproduce the Problem

  1. I've slightly edited the example binance_futures_testnet_sandbox.py:
#!/usr/bin/env python3
# -------------------------------------------------------------------------------------------------
#  Copyright (C) 2015-2024 Nautech Systems Pty Ltd. All rights reserved.
#  https://nautechsystems.io
#
#  Licensed under the GNU Lesser General Public License Version 3.0 (the "License");
#  You may not use this file except in compliance with the License.
#  You may obtain a copy of the License at https://www.gnu.org/licenses/lgpl-3.0.en.html
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.
# -------------------------------------------------------------------------------------------------

import asyncio
from decimal import Decimal

from nautilus_trader.adapters.binance.common.enums import BinanceAccountType
from nautilus_trader.adapters.binance.config import BinanceDataClientConfig
from nautilus_trader.adapters.binance.factories import BinanceLiveDataClientFactory
from nautilus_trader.adapters.sandbox.config import SandboxExecutionClientConfig
from nautilus_trader.adapters.sandbox.factory import SandboxLiveExecClientFactory
from nautilus_trader.config import CacheConfig
from nautilus_trader.config import InstrumentProviderConfig
from nautilus_trader.config import LiveExecEngineConfig
from nautilus_trader.config import LoggingConfig
from nautilus_trader.config import TradingNodeConfig
from nautilus_trader.examples.strategies.volatility_market_maker import VolatilityMarketMaker
from nautilus_trader.examples.strategies.volatility_market_maker import VolatilityMarketMakerConfig
from nautilus_trader.live.node import TradingNode
from nautilus_trader.model.data import BarType
from nautilus_trader.model.identifiers import InstrumentId
from nautilus_trader.model.identifiers import TraderId

import os
import dotenv

dotenv.load_dotenv()

# *** THIS IS A TEST STRATEGY WITH NO ALPHA ADVANTAGE WHATSOEVER. ***
# *** IT IS NOT INTENDED TO BE USED TO TRADE LIVE WITH REAL MONEY. ***


async def main():
    """
    Show how to run a strategy in a sandbox for the Binance venue.
    """
    # Configure the trading node
    config_node = TradingNodeConfig(
        trader_id=TraderId("TESTER-001"),
        logging=LoggingConfig(
            log_level="INFO",
            # log_level_file="DEBUG",
            # log_file_format="json",
            log_colors=True,
            use_pyo3=True,
        ),
        exec_engine=LiveExecEngineConfig(
            reconciliation=True,
            reconciliation_lookback_mins=1440,
            filter_position_reports=True,
            # snapshot_orders=True,
            # snapshot_positions=True,
            # snapshot_positions_interval_secs=5.0,
        ),
        cache=CacheConfig(
            # database=DatabaseConfig(timeout=2),
            timestamps_as_iso8601=True,
            flush_on_start=False,
        ),
        # message_bus=MessageBusConfig(
        #     database=DatabaseConfig(timeout=2),
        #     encoding="json",
        #     timestamps_as_iso8601=True,
        #     streams_prefix="quoters",
        #     use_instance_id=False,
        #     # types_filter=[QuoteTick],
        #     autotrim_mins=30,
        #     heartbeat_interval_secs=1,
        # ),
        data_clients={
            "BINANCE": BinanceDataClientConfig(
                api_key=os.getenv("BINANCE_API_KEY"),  # 'BINANCE_API_KEY' env var
                api_secret=os.getenv("BINANCE_API_SECRET"),  # 'BINANCE_API_SECRET' env var
                account_type=BinanceAccountType.MARGIN,
                base_url_http=None,  # Override with custom endpoint
                base_url_ws=None,  # Override with custom endpoint
                us=False,  # If client is for Binance US
                testnet=False,  # If client uses the testnet
                instrument_provider=InstrumentProviderConfig(load_all=True),
            ),
        },
        exec_clients={
            "BINANCE": SandboxExecutionClientConfig(
                venue="BINANCE",
                starting_balances=["10_000 USDT", "10 ETH"],
            ),
        },
        timeout_connection=30.0,
        timeout_reconciliation=10.0,
        timeout_portfolio=10.0,
        timeout_disconnection=10.0,
        timeout_post_stop=5.0,
    )

    # Instantiate the node with a configuration
    node = TradingNode(config=config_node)

    # Configure your strategy
    strat_config = VolatilityMarketMakerConfig(
        instrument_id=InstrumentId.from_str("ETHUSDT.BINANCE"),
        external_order_claims=[InstrumentId.from_str("ETHUSDT.BINANCE")],
        bar_type=BarType.from_str("ETHUSDT.BINANCE-1-MINUTE-LAST-EXTERNAL"),
        atr_period=20,
        atr_multiple=6.0,
        trade_size=Decimal("0.010"),
        # manage_gtd_expiry=True,
    )
    # Instantiate your strategy
    strategy = VolatilityMarketMaker(config=strat_config)

    # Add your strategies and modules
    node.trader.add_strategy(strategy)

    # Register your client factories with the node (can take user-defined factories)
    node.add_data_client_factory("BINANCE", BinanceLiveDataClientFactory)
    node.add_exec_client_factory("BINANCE", SandboxLiveExecClientFactory)
    node.build()

    try:
        await node.run_async()
    finally:
        await node.stop_async()
        await asyncio.sleep(1)
        node.dispose()


# Stop and dispose of the node with SIGINT/CTRL+C
if __name__ == "__main__":
    asyncio.run(main())
  1. Run.
  2. Changing account type to SPOT or setting testnet=True works. Not sure if there's a misconfig on my side.

Full error log

2024-12-17T21:38:30.135079001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.135689001Z [INFO] TESTER-001.TradingNode:  NAUTILUS TRADER - Automated Algorithmic Trading Platform
2024-12-17T21:38:30.135694001Z [INFO] TESTER-001.TradingNode:  by Nautech Systems Pty Ltd.
2024-12-17T21:38:30.135696001Z [INFO] TESTER-001.TradingNode:  Copyright (C) 2015-2024. All rights reserved.
2024-12-17T21:38:30.135698001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.135699001Z [INFO] TESTER-001.TradingNode: 
2024-12-17T21:38:30.135701001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⣴⣶⡟⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
2024-12-17T21:38:30.135703001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣰⣾⣿⣿⣿⠀⢸⣿⣿⣿⣿⣶⣶⣤⣀⠀⠀⠀⠀⠀
2024-12-17T21:38:30.135905001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠀⠀⠀⢀⣴⡇⢀⣾⣿⣿⣿⣿⣿⠀⣾⣿⣿⣿⣿⣿⣿⣿⠿⠓⠀⠀⠀⠀
2024-12-17T21:38:30.135912001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠀⠀⣰⣿⣿⡀⢸⣿⣿⣿⣿⣿⣿⠀⣿⣿⣿⣿⣿⣿⠟⠁⣠⣄⠀⠀⠀⠀
2024-12-17T21:38:30.135914001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠀⢠⣿⣿⣿⣇⠀⢿⣿⣿⣿⣿⣿⠀⢻⣿⣿⣿⡿⢃⣠⣾⣿⣿⣧⡀⠀⠀
2024-12-17T21:38:30.135916001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠠⣾⣿⣿⣿⣿⣿⣧⠈⠋⢀⣴⣧⠀⣿⡏⢠⡀⢸⣿⣿⣿⣿⣿⣿⣿⡇⠀
2024-12-17T21:38:30.135917001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⣀⠙⢿⣿⣿⣿⣿⣿⠇⢠⣿⣿⣿⡄⠹⠃⠼⠃⠈⠉⠛⠛⠛⠛⠛⠻⠇⠀
2024-12-17T21:38:30.135919001Z [INFO] TESTER-001.TradingNode: ⠀⠀⢸⡟⢠⣤⠉⠛⠿⢿⣿⠀⢸⣿⡿⠋⣠⣤⣄⠀⣾⣿⣿⣶⣶⣶⣦⡄⠀⠀⠀
2024-12-17T21:38:30.136079001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠸⠀⣾⠏⣸⣷⠂⣠⣤⠀⠘⢁⣴⣾⣿⣿⣿⡆⠘⣿⣿⣿⣿⣿⣿⠀⠀⠀⠀
2024-12-17T21:38:30.136084001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠀⠛⠀⣿⡟⠀⢻⣿⡄⠸⣿⣿⣿⣿⣿⣿⣿⡀⠘⣿⣿⣿⣿⠟⠀⠀⠀⠀
2024-12-17T21:38:30.136086001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⢻⡿⠀⠈⠻⣿⣿⣿⣿⣿⡇⠀⢹⣿⠿⠋⠀⠀⠀⠀⠀
2024-12-17T21:38:30.136106001Z [INFO] TESTER-001.TradingNode: ⠀⠀⠀⠀⠀⠀⠋⠀⠀⠀⡘⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠁⠀⠀⠀⠀⠀⠀⠀
2024-12-17T21:38:30.136110001Z [INFO] TESTER-001.TradingNode: 
2024-12-17T21:38:30.136112001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.136113001Z [INFO] TESTER-001.TradingNode:  SYSTEM SPECIFICATION
2024-12-17T21:38:30.136144001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.136150001Z [INFO] TESTER-001.TradingNode: CPU architecture: Apple M1 Pro
2024-12-17T21:38:30.136152001Z [INFO] TESTER-001.TradingNode: CPU(s): 10 @ 3228 Mhz
2024-12-17T21:38:30.136153001Z [INFO] TESTER-001.TradingNode: OS: kernel-24.1.0 MacOS 15.1.1 
2024-12-17T21:38:30.144072001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.144089001Z [INFO] TESTER-001.TradingNode:  MEMORY USAGE
2024-12-17T21:38:30.144091001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.144307001Z [INFO] TESTER-001.TradingNode: RAM-Total: 16.00 GiB
2024-12-17T21:38:30.144312001Z [INFO] TESTER-001.TradingNode: RAM-Used: 12.28 GiB (76.76%)
2024-12-17T21:38:30.144314001Z [INFO] TESTER-001.TradingNode: RAM-Avail: 3.72 GiB (23.24%)
2024-12-17T21:38:30.144316001Z [INFO] TESTER-001.TradingNode: Swap-Total: 23.00 GiB
2024-12-17T21:38:30.144317001Z [INFO] TESTER-001.TradingNode: Swap-Used: 21.89 GiB (95.16%)
2024-12-17T21:38:30.144318001Z [INFO] TESTER-001.TradingNode: Swap-Avail: 1.11 GiB (4.84%)
2024-12-17T21:38:30.144759001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.144765001Z [INFO] TESTER-001.TradingNode:  IDENTIFIERS
2024-12-17T21:38:30.144767001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.144768001Z [INFO] TESTER-001.TradingNode: trader_id: TESTER-001
2024-12-17T21:38:30.144770001Z [INFO] TESTER-001.TradingNode: machine_id: 192.168.1.233
2024-12-17T21:38:30.144771001Z [INFO] TESTER-001.TradingNode: instance_id: d9deabf4-8f6a-4b8c-9b24-97249594160a
2024-12-17T21:38:30.144772001Z [INFO] TESTER-001.TradingNode: PID: 8002
2024-12-17T21:38:30.144773001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.144775001Z [INFO] TESTER-001.TradingNode:  VERSIONING
2024-12-17T21:38:30.144788001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.144796001Z [INFO] TESTER-001.TradingNode: nautilus_trader: latest
2024-12-17T21:38:30.144797001Z [INFO] TESTER-001.TradingNode: python: 3.12.8
2024-12-17T21:38:30.144800001Z [INFO] TESTER-001.TradingNode: numpy: 2.2.0
2024-12-17T21:38:30.144802001Z [INFO] TESTER-001.TradingNode: pandas: 2.2.3
2024-12-17T21:38:30.144805001Z [INFO] TESTER-001.TradingNode: msgspec: 0.18.6
2024-12-17T21:38:30.144806001Z [INFO] TESTER-001.TradingNode: pyarrow: 18.1.0
2024-12-17T21:38:30.144808001Z [INFO] TESTER-001.TradingNode: pytz: 2024.2
2024-12-17T21:38:30.144817001Z [INFO] TESTER-001.TradingNode: uvloop: 0.20.0
2024-12-17T21:38:30.144820001Z [INFO] TESTER-001.TradingNode: =================================================================
2024-12-17T21:38:30.146774001Z [INFO] TESTER-001.TradingNode: Building system kernel
2024-12-17T21:38:30.147253001Z [INFO] TESTER-001.MessageBus: config.database=None
2024-12-17T21:38:30.147260001Z [INFO] TESTER-001.MessageBus: config.encoding='msgpack'
2024-12-17T21:38:30.147261001Z [INFO] TESTER-001.MessageBus: config.timestamps_as_iso8601=False
2024-12-17T21:38:30.147263001Z [INFO] TESTER-001.MessageBus: config.buffer_interval_ms=None
2024-12-17T21:38:30.147264001Z [INFO] TESTER-001.MessageBus: config.autotrim_mins=None
2024-12-17T21:38:30.147266001Z [INFO] TESTER-001.MessageBus: config.use_trader_prefix=True
2024-12-17T21:38:30.147267001Z [INFO] TESTER-001.MessageBus: config.use_trader_id=True
2024-12-17T21:38:30.147268001Z [INFO] TESTER-001.MessageBus: config.use_instance_id=False
2024-12-17T21:38:30.147270001Z [INFO] TESTER-001.MessageBus: config.streams_prefix='stream'
2024-12-17T21:38:30.147271001Z [INFO] TESTER-001.MessageBus: config.types_filter=None
2024-12-17T21:38:30.147604001Z [INFO] TESTER-001.Cache: READY
2024-12-17T21:38:30.149677001Z [INFO] TESTER-001.DataEngine: READY
2024-12-17T21:38:30.149891001Z [INFO] TESTER-001.RiskEngine: READY
2024-12-17T21:38:30.151014001Z [INFO] TESTER-001.RiskEngine: TradingState is ACTIVE
2024-12-17T21:38:30.151106001Z [INFO] TESTER-001.Throttler-ORDER_SUBMIT_THROTTLER: READY
2024-12-17T21:38:30.151128001Z [INFO] TESTER-001.RiskEngine: Set MAX_ORDER_SUBMIT_RATE: 100/00:00:01
2024-12-17T21:38:30.151193001Z [INFO] TESTER-001.Throttler-ORDER_MODIFY_THROTTLER: READY
2024-12-17T21:38:30.151209001Z [INFO] TESTER-001.RiskEngine: Set MAX_ORDER_MODIFY_RATE: 100/00:00:01
2024-12-17T21:38:30.151555001Z [INFO] TESTER-001.ExecEngine: READY
2024-12-17T21:38:30.151782001Z [INFO] TESTER-001.ExecEngine: config.snapshot_orders=False
2024-12-17T21:38:30.151787001Z [INFO] TESTER-001.ExecEngine: config.snapshot_positions=False
2024-12-17T21:38:30.151789001Z [INFO] TESTER-001.ExecEngine: config.snapshot_positions_interval_secs=None
2024-12-17T21:38:30.151827001Z [INFO] TESTER-001.ExecEngine: config.reconciliation=True
2024-12-17T21:38:30.151844001Z [INFO] TESTER-001.ExecEngine: config.reconciliation_lookback_mins=1440
2024-12-17T21:38:30.151847001Z [INFO] TESTER-001.ExecEngine: config.filter_unclaimed_external_orders=False
2024-12-17T21:38:30.151849001Z [INFO] TESTER-001.ExecEngine: config.filter_position_reports=True
2024-12-17T21:38:30.151850001Z [INFO] TESTER-001.ExecEngine: config.inflight_check_interval_ms=2000
2024-12-17T21:38:30.151852001Z [INFO] TESTER-001.ExecEngine: config.inflight_check_threshold_ms=5000
2024-12-17T21:38:30.151854001Z [INFO] TESTER-001.ExecEngine: config.inflight_check_retries=5
2024-12-17T21:38:30.152423001Z [INFO] TESTER-001.Cache: Cached 0 general objects from database
2024-12-17T21:38:30.152459001Z [INFO] TESTER-001.Cache: Cached 0 currencies from database
2024-12-17T21:38:30.152467001Z [INFO] TESTER-001.Cache: Cached 0 instruments from database
2024-12-17T21:38:30.152469001Z [INFO] TESTER-001.Cache: Cached 0 accounts from database
2024-12-17T21:38:30.152471001Z [INFO] TESTER-001.Cache: Cached 0 orders from database
2024-12-17T21:38:30.152472001Z [INFO] TESTER-001.Cache: Cached 0 order lists from database
2024-12-17T21:38:30.152473001Z [INFO] TESTER-001.Cache: Cached 0 positions from database
2024-12-17T21:38:30.152675001Z [INFO] TESTER-001.Cache: Checking data integrity
2024-12-17T21:38:30.152715001Z [INFO] TESTER-001.Cache: Integrity check passed in 40μs
2024-12-17T21:38:30.153063001Z [INFO] TESTER-001.ExecEngine: Loaded cache in 1ms
2024-12-17T21:38:30.154092001Z [INFO] TESTER-001.OrderEmulator: READY
2024-12-17T21:38:30.154191001Z [INFO] TESTER-001.TESTER-001: READY
2024-12-17T21:38:30.154227001Z [INFO] TESTER-001.TradingNode: Initialized in 30ms
2024-12-17T21:38:30.154237001Z [INFO] TESTER-001.TradingNode: self._has_cache_backing=None
2024-12-17T21:38:30.154239001Z [INFO] TESTER-001.TradingNode: self._has_msgbus_backing=None
2024-12-17T21:38:30.158896001Z [INFO] TESTER-001.VolatilityMarketMaker: READY
2024-12-17T21:38:30.159531001Z [INFO] TESTER-001.ExecEngine: Registered OMS.UNSPECIFIED for Strategy VolatilityMarketMaker-000
2024-12-17T21:38:30.159538001Z [INFO] TESTER-001.ExecEngine: Registered external order claims for VolatilityMarketMaker-000: [InstrumentId('ETHUSDT.BINANCE')]
2024-12-17T21:38:30.159543001Z [INFO] TESTER-001.TESTER-001: Registered Strategy VolatilityMarketMaker-000
2024-12-17T21:38:30.159584001Z [INFO] TESTER-001.TradingNode: Building data client for BINANCE
2024-12-17T21:38:30.160888001Z [INFO] TESTER-001.BinanceSpotInstrumentProvider: READY
2024-12-17T21:38:30.162439001Z [INFO] TESTER-001.DataClient-BINANCE: READY
2024-12-17T21:38:30.162557001Z [INFO] TESTER-001.DataClient-BINANCE: Key type: HMAC
2024-12-17T21:38:30.162567001Z [INFO] TESTER-001.DataClient-BINANCE: Account type: MARGIN
2024-12-17T21:38:30.162569001Z [INFO] TESTER-001.DataClient-BINANCE: config.use_agg_trade_ticks=False
2024-12-17T21:38:30.162577001Z [INFO] TESTER-001.DataClient-BINANCE: Base url HTTP https://sapi.binance.com
2024-12-17T21:38:30.162578001Z [INFO] TESTER-001.DataClient-BINANCE: Base url WebSocket wss://stream.binance.com:9443
2024-12-17T21:38:30.162788001Z [INFO] TESTER-001.DataEngine: Registered BINANCE
2024-12-17T21:38:30.162795001Z [INFO] TESTER-001.TradingNode: Building execution client for BINANCE
2024-12-17T21:38:30.163528001Z [INFO] TESTER-001.InstrumentProvider: READY
2024-12-17T21:38:30.163563001Z [INFO] TESTER-001.ExecClient-BINANCE: READY
2024-12-17T21:38:30.168364001Z [INFO] TESTER-001.SimulatedExchange(BINANCE): OmsType=NETTING
2024-12-17T21:38:30.168399001Z [INFO] TESTER-001.ExecClient-BINANCE: READY
2024-12-17T21:38:30.168450001Z [INFO] TESTER-001.SimulatedExchange(BINANCE): Registered ExecutionClient-BINANCE
2024-12-17T21:38:30.169855001Z [INFO] TESTER-001.Portfolio: Updated AccountState(account_id=BINANCE-001, account_type=MARGIN, base_currency=None, is_reported=True, balances=[AccountBalance(total=10_000.00000000 USDT, locked=0.00000000 USDT, free=10_000.00000000 USDT), AccountBalance(total=10.00000000 ETH, locked=0.00000000 ETH, free=10.00000000 ETH)], margins=[], event_id=97568c44-d2de-4e6c-951c-ab9b3a27f145)
2024-12-17T21:38:30.170055001Z [INFO] TESTER-001.ExecEngine: Registered ExecutionClient-BINANCE
2024-12-17T21:38:30.170084001Z [INFO] TESTER-001.TradingNode: STARTING
2024-12-17T21:38:30.170224001Z [INFO] TESTER-001.DataClient-BINANCE: RUNNING
2024-12-17T21:38:30.170449001Z [INFO] TESTER-001.DataEngine: RUNNING
2024-12-17T21:38:30.170474001Z [INFO] TESTER-001.RiskEngine: RUNNING
2024-12-17T21:38:30.170494001Z [INFO] TESTER-001.ExecClient-BINANCE: RUNNING
2024-12-17T21:38:30.170512001Z [INFO] TESTER-001.ExecEngine: RUNNING
2024-12-17T21:38:30.170552001Z [INFO] TESTER-001.DataEngine: Connecting all clients...
2024-12-17T21:38:30.170555001Z [INFO] TESTER-001.DataClient-BINANCE: Connecting...
2024-12-17T21:38:30.170652001Z [INFO] TESTER-001.ExecEngine: Connecting all clients...
2024-12-17T21:38:30.170681001Z [INFO] TESTER-001.ExecClient-BINANCE: Connecting...
2024-12-17T21:38:30.170749001Z [INFO] TESTER-001.ExecClient-BINANCE: Connected
2024-12-17T21:38:30.170756001Z [INFO] TESTER-001.TradingNode: Awaiting engine connections and initializations (30.0s timeout)...
2024-12-17T21:38:30.170928001Z [INFO] TESTER-001.DataClient-BINANCE: Initializing instruments...
2024-12-17T21:38:30.170932001Z [INFO] TESTER-001.BinanceSpotInstrumentProvider: Loading all instruments...
2024-12-17T21:38:32.694659001Z [ERROR] TESTER-001.DataClient-BINANCE: Error on '_connect': DecodeError('JSON is malformed: invalid character (byte 0)')
Traceback (most recent call last):
  File "{REDACTED}/.venv/lib/python3.12/site-packages/nautilus_trader/adapters/binance/data.py", line 216, in _connect
    await self._instrument_provider.initialize()
  File "{REDACTED}/.venv/lib/python3.12/site-packages/nautilus_trader/common/providers.py", line 152, in initialize
    await self.load_all_async(self._filters)
  File "{REDACTED}/venv/lib/python3.12/site-packages/nautilus_trader/adapters/binance/spot/providers.py", line 104, in load_all_async
    response = await self._http_wallet.query_spot_trade_fees()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "{REDACTED}/.venv/lib/python3.12/site-packages/nautilus_trader/adapters/binance/spot/http/wallet.py", line 122, in query_spot_trade_fees
    fees = await self._endpoint_spot_trade_fee.get(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "{REDACTED}/.venv/lib/python3.12/site-packages/nautilus_trader/adapters/binance/spot/http/wallet.py", line 81, in get
    return self._get_arr_resp_decoder.decode(raw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
msgspec.DecodeError: JSON is malformed: invalid character (byte 0)

I've tried to debug the response it seems to be a 200 but couldn't figure out how to decode the bytes.

Specifications

  • OS platform: MacOS 15.1.1
  • Python version: 3.11
  • nautilus_trader version: 1.202.0
@italodamato italodamato added the bug Something isn't working label Dec 17, 2024
@cjdsellers
Copy link
Member

Hi @italodamato

Thanks for the report.

The latest released version is 1.208.0, and there's a high chance this has been fixed in that version.
Please report if you still encounter the issue after upgrading.

@cjdsellers cjdsellers removed the bug Something isn't working label Dec 17, 2024
@italodamato
Copy link
Author

italodamato commented Dec 17, 2024

Thanks for replying @cjdsellers. Just updated and still having the problem. I did more digging and it seems to be related to the base URL being sapi.binance.com but it should just be api.binance.com. Also found this related issue.

@italodamato
Copy link
Author

italodamato commented Dec 17, 2024

I think it might be because I'm in the UK and they redirect me to another domain, which returns an HTML instead of a JSON.

2024-12-17T23:45:00.616918001Z [DEBUG] TESTER-001.reqwest::connect: starting new connection: https://sapi.binance.com/
2024-12-17T23:45:01.564509001Z [DEBUG] TESTER-001.reqwest::async_impl::client: redirecting 'https://sapi.binance.com/sapi/v1/asset/tradeFee?timestamp=1734479100615&signature={REDACTED}' to 'https://www.binance.com/en'
2024-12-17T23:45:01.564784001Z [DEBUG] TESTER-001.reqwest::connect: starting new connection: https://www.binance.com/
2024-12-17T23:45:01.818783001Z [DEBUG] TESTER-001.reqwest::async_impl::client: redirecting 'https://www.binance.com/en' to 'https://www.binance.com/en-GB'

getting this when selecting SPOT

2024-12-17T23:44:28.793607001Z [DEBUG] TESTER-001.reqwest::connect: starting new connection: https://api.binance.com/

@cjdsellers
Copy link
Member

Thanks for the update @italodamato

Right now there's limited bandwidth to work on supporting Binance margin. Looking at the related issue, I think it goes deeper than just message parsing, but relates to how we calculate margin for accounts.

@italodamato
Copy link
Author

Thank you for the additional context, @cjdsellers.

Could you provide a list of fully implemented use cases (e.g., adapters and account types)? This would help potential new users decide if it's the right time to start building on this framework.

I had assumed Binance was already fully supported since it's listed as green and stable in the README. Additionally, the roadmap seems to focus more on refactoring rather than adding new features, which further reinforces that assumption.

@cjdsellers
Copy link
Member

Hi @italodamato

The definitions for integration status can be found here. For stable, we define this as:

Stabilized feature set and API, the integration has been tested by both developers and users to a reasonable level (some bugs may still remain).

In the case of Binance Spot and Futures, these integrations were used in production by multiple users for months before we marked them as stable. However, this doesn’t imply that the integration is feature-complete. The Binance API is particularly extensive, with numerous variations, such as isolated and cross-margin handling, and dedicated endpoints for margin functionality.

The implementation goals for integrations state:

The full range of an exchanges functionality (where applicable to NautilusTrader), should eventually be supported.

At the moment, Binance's margin API is not as well-supported or tested due to limited bandwidth among maintainers and contributors. So if Binance margin is critical for your use case, the platform may not yet fully meet your needs at this point in time.

That said, contributions or feedback are always welcome, as they help us prioritize features that matter most to users.

@italodamato
Copy link
Author

italodamato commented Dec 19, 2024

Thanks for the explanation, that's really helpful. Would you be able to expand on the table in the main README where, for each component and adaptor, we make explicit what's fully/partially functional/missing?

For instance (made this up):

Adapter Fully functional Partially functional Missing
Binance -Spot
-Futures
- Margin mode (endpoints breaking)
- Isolated-margin (...)
...
Bybit ... ... ...
Component Notes
Backtesting - Funding rates are not considered in the calculations
Data ...
Execution ...

I think having the unknown unknowns (i.e. the gotchas) clear upfront would be great for users looking to adopt the framework while still in beta. This is especially important for non-breaking and silently missing features like assumptions and missing calculations in the backtesting engine.

If this gets too complicated, a simpler way would be to create a backlog ticket for each of these to make them explicit for people to refer to, increasing transparency and potential community contributions.

@cjdsellers
Copy link
Member

Hi @italodamato

Thanks for the suggestions! I think this would definitely be a clearer way to lay things out for users and potential users.

@cjdsellers cjdsellers changed the title msgspec.DecodeError on Binance Margin Wallet TradeFee - sandbox Document capabilities with more clarity Dec 22, 2024
@cjdsellers cjdsellers self-assigned this Dec 22, 2024
@cjdsellers cjdsellers added the docs Relating to documentation label Dec 22, 2024
@cjdsellers cjdsellers moved this to In progress in NautilusTrader Kanban Board Dec 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Relating to documentation
Projects
Status: In progress
Development

No branches or pull requests

2 participants