Algo Descriptors

Algo descriptors define the algorithms that will run in your Datafye environment. They specify the container image, parameters, resource requirements, and optimization space for your trading strategies.

Purpose

Algo descriptors tell Datafye:

What container image hosts your algo
What parameters configure your algo's behavior
What parameter space to explore during optimization
What resources your algo needs (CPU, memory, GPU)
What inputs your algo consumes (data feeds, signals)
What outputs your algo produces (signals, metrics, trades)

Based on your algo descriptor, Datafye provisions the Algo Container runtime and configures it to execute your strategy.

When You Need Algo Descriptors

Algo descriptors are required for Full Stack scenarios only:

Foundry: Full Stack — Defines algos for backtesting and optimization
Trading: Full Stack — Configures algos for paper and live trading

For Data Cloud Only and Data Cloud + Broker scenarios, you provide your own containers and don't use algo descriptors.

Structure

TODO: Provide actual YAML/JSON schema for algo descriptors with complete field definitions.

A typical algo descriptor includes:

algo:
  name: "my-strategy"
  image: "myrepo/strategy:v1.0"

  parameters:
    # Fixed parameter values

  parameter_space:
    # Parameter ranges for optimization

  resources:
    # CPU, memory, GPU requirements

  inputs:
    # Data feeds and signals consumed

  outputs:
    # Signals and metrics produced

Basic Configuration

Algo Name

A unique identifier for your algo:

algo:
  name: "momentum-strategy"
  # [Naming requirements? Length limits? Allowed characters?]

Best practices:

Use descriptive names that indicate strategy type
Include version in name for clarity (momentum-v2)
Use lowercase with hyphens (my-algo, not MyAlgo or my_algo)

Container Image

The Docker image containing your algo implementation:

algo:
  name: "momentum-strategy"
  image: "myrepo/momentum:v1.0"
  # [Image registry requirements? Public vs. private?]
  # [Authentication for private registries?]

Image requirements:

Built with Datafye Algo Container SDK
Implements required SDK interfaces
Includes all dependencies
Tagged with specific version (not latest)

Registry options:

Docker Hub: username/image:tag
Private registry: registry.example.com/image:tag
[Does Datafye provide a container registry?]

Parameters

Fixed parameter values that configure your algo:

algo:
  name: "momentum-strategy"
  image: "myrepo/momentum:v1.0"

  parameters:
    lookback_period: 20
    threshold: 0.02
    stop_loss: 0.05
    take_profit: 0.10
    position_size: 0.1
    # [Data types supported? int, float, string, bool, arrays?]

Parameter types:

Numeric: integers and floats
Strings: text values
Booleans: true/false flags
[Arrays/lists supported?]
[Nested objects supported?]

Use cases:

Strategy configuration (lookback periods, thresholds)
Risk management (stop loss, position sizing)
Execution settings (order types, time in force)
Feature flags (enable/disable components)

Parameter Space (For Optimization)

The parameter space defines ranges for genetic algorithm-based optimization:

algo:
  name: "momentum-strategy"
  image: "myrepo/momentum:v1.0"

  parameters:
    lookback_period: 20
    threshold: 0.02

  parameter_space:
    lookback_period: [10, 50]      # Search from 10 to 50
    threshold: [0.01, 0.05]        # Search from 0.01 to 0.05
    # [Step size configurable?]
    # [Discrete vs. continuous ranges?]

Range Specification

TODO: Document exact syntax for parameter space ranges, step sizes, discrete vs. continuous, and any constraints.

Continuous ranges:

parameter_space:
  threshold: [0.01, 0.10]  # Any value between 0.01 and 0.10
  # [Uniform sampling? Log scale option?]

Discrete values:

parameter_space:
  lookback_period: [10, 20, 30, 50]  # Only these specific values
  # Or using range with step
  lookback_period:
    min: 10
    max: 50
    step: 5  # [Is this syntax supported?]

Categorical parameters:

parameter_space:
  signal_type: ["momentum", "mean_reversion", "breakout"]
  # [Are categorical parameters supported?]

Optimization Strategy

The genetic algorithm explores the parameter space to find optimal combinations:

algo:
  name: "momentum-strategy"
  image: "myrepo/momentum:v1.0"

  parameter_space:
    lookback_period: [10, 50]
    threshold: [0.01, 0.05]

  optimization:
    objective: "sharpe_ratio"  # [Available objectives?]
    generations: 50            # [How many generations to run?]
    population_size: 100       # [How many candidates per generation?]
    # [Other optimization settings?]

TODO: Document optimization configuration options, objectives, and constraints.

Optimization objectives:

sharpe_ratio — Maximize risk-adjusted returns
total_return — Maximize absolute returns
sortino_ratio — Maximize downside-adjusted returns
calmar_ratio — Maximize return/drawdown ratio
[Other objectives?]
[Multi-objective optimization supported?]

Constraints:

[Can you set constraints like max_drawdown < 0.2?]
[Can you constrain parameter relationships?]
[Can you set resource limits per backtest?]

Resource Requirements

Specify compute resources your algo needs:

algo:
  name: "momentum-strategy"
  image: "myrepo/momentum:v1.0"

  resources:
    cpu: 2              # Number of CPU cores
    memory: "4Gi"       # Memory allocation
    gpu: 0              # GPU count (0 = none)
    # [Disk space configurable?]
    # [Network bandwidth?]

CPU Allocation

resources:
  cpu: 2  # 2 cores
  # [Fractional cores supported? e.g., 0.5?]
  # [CPU limits vs. requests?]

Considerations:

More CPUs = faster backtesting (if algo is parallelizable)
Optimize based on actual usage
[Cost implications of CPU allocation?]

Memory Allocation

resources:
  memory: "4Gi"  # 4 gigabytes
  # [Units: Gi (gibibytes), Mi (mebibytes)?]
  # [Memory limits vs. requests?]

Considerations:

Depends on data volume and algo complexity
Monitor actual usage and adjust
Out-of-memory = algo crashes
[Memory limits enforced strictly?]

GPU Support

resources:
  gpu: 1  # Request 1 GPU
  gpu_type: "nvidia-tesla-t4"  # [Specific GPU types?]
  # [GPU memory configurable?]

Use cases:

Machine learning models
Large-scale computations
Real-time inference

Considerations:

[Are GPUs available in all deployment types?]
[Significant cost implications?]
Not all algos need GPUs

Inputs and Outputs

Define what data your algo consumes and produces:

TODO: Document the input/output specification format and how it integrates with the SDK.

Inputs

algo:
  name: "momentum-strategy"
  image: "myrepo/momentum:v1.0"

  inputs:
    - type: "market_data"
      symbols: ["AAPL", "GOOGL"]
      fields: ["open", "high", "low", "close", "volume"]

    - type: "signal"
      source: "another-algo"
      # [Can algos consume signals from other algos?]

    - type: "fundamental_data"
      provider: "datafye_fundamental"
      # [How are alternative inputs specified?]

Input types:

Market data (OHLCV, quotes, trades)
Signals from other algos (composability)
Alternative data (sentiment, fundamentals)
[Custom data sources?]

Outputs

algo:
  name: "momentum-strategy"
  image: "myrepo/momentum:v1.0"

  outputs:
    - type: "signal"
      name: "momentum_score"
      description: "Momentum strength from -1 to 1"

    - type: "trade"
      description: "Buy/sell orders"

    - type: "metric"
      name: "signal_confidence"
      description: "Confidence level 0-1"

Output types:

Signals (for other algos or visualization)
Trades (actual orders)
Metrics (for monitoring and analysis)
[Alerts/notifications?]

Example Algo Descriptors

Example 1: Simple Momentum Strategy

Basic momentum algo with fixed parameters:

algo:
  name: "simple-momentum"
  image: "myrepo/momentum:v1.0"

  parameters:
    lookback_period: 20
    threshold: 0.02

  resources:
    cpu: 1
    memory: "2Gi"

Example 2: Optimizable Strategy

Momentum strategy with parameter space for optimization:

algo:
  name: "momentum-optimized"
  image: "myrepo/momentum:v1.0"

  parameters:
    lookback_period: 20
    threshold: 0.02
    stop_loss: 0.05

  parameter_space:
    lookback_period: [10, 50]
    threshold: [0.01, 0.05]
    stop_loss: [0.03, 0.10]

  optimization:
    objective: "sharpe_ratio"
    generations: 50
    population_size: 100

  resources:
    cpu: 2
    memory: "4Gi"

Example 3: ML-Based Strategy

Machine learning algo requiring GPU:

algo:
  name: "ml-predictor"
  image: "myrepo/ml-strategy:v2.0"

  parameters:
    model_path: "/models/predictor.h5"
    lookback_bars: 100
    confidence_threshold: 0.7

  resources:
    cpu: 4
    memory: "16Gi"
    gpu: 1
    gpu_type: "nvidia-tesla-t4"

Example 4: Composable Strategy

Algo that consumes signals from other algos:

algo:
  name: "ensemble-strategy"
  image: "myrepo/ensemble:v1.0"

  inputs:
    - type: "signal"
      source: "momentum-strategy"
      field: "momentum_score"

    - type: "signal"
      source: "mean-reversion-strategy"
      field: "reversion_score"

  parameters:
    momentum_weight: 0.6
    reversion_weight: 0.4

  outputs:
    - type: "trade"
      description: "Combined buy/sell signals"

  resources:
    cpu: 1
    memory: "2Gi"

SDK Integration

Algo descriptors work hand-in-hand with the Datafye Algo Container SDK:

Parameter Access

Your SDK-based algo accesses parameters defined in the descriptor:

# In your algo code (example - actual SDK API may differ)
from datafye.sdk import AlgoContainer

class MomentumStrategy(AlgoContainer):
    def initialize(self):
        # Access parameters from descriptor
        self.lookback = self.params.get('lookback_period')
        self.threshold = self.params.get('threshold')

Resource Utilization

The SDK ensures your algo stays within resource limits:

# SDK handles resource constraints automatically
# No need to manually manage CPU/memory limits

Input/Output Handling

The SDK provides APIs for consuming inputs and producing outputs:

def on_bar(self, bar):
    # SDK delivers market data based on inputs in descriptor
    price = bar.close

    # Calculate signal
    signal = self.calculate_momentum(price)

    # Emit output (SDK handles based on outputs in descriptor)
    self.emit_signal('momentum_score', signal)

Validation

Before provisioning, Datafye validates your algo descriptor:

Schema Validation

All required fields present
Valid parameter types and values
Resource requirements within limits
Image reference format correct

Image Validation

Container image exists and is accessible
Image built with compatible SDK version
Image passes basic health checks
[Size limits on images?]

Parameter Validation

Parameters match algo's expected config
Parameter space ranges are valid
No conflicting parameter combinations
[Does SDK declare expected parameters for validation?]

Resource Validation

Resource requests are reasonable
Resources available in deployment environment
[Maximum resource limits?]

Common validation errors:

Container image not found or inaccessible
Invalid parameter types
Parameter space range errors
Resource requests exceed limits
[Other common errors?]

Versioning and Updates

Algo Versioning

Use image tags to version your algos:

algo:
  name: "momentum-strategy"
  image: "myrepo/momentum:v1.0"  # Specific version

Best practices:

Use semantic versioning (v1.0.0, v1.1.0, v2.0.0)
Never use latest tag in production
Tag images with git commit hash for traceability
Document breaking changes between versions

Updating Algos

To update an algo in a running environment:

Build new container image with changes
Tag with new version
Update algo descriptor with new image reference
Re-provision or restart environment

[Does Datafye support hot-swapping algos without full reprovision?]

Development Workflow

Local Development

Develop algo using SDK
- Implement strategy logic
- Define parameters and inputs/outputs
- Test locally with sample data
Containerize
- Write Dockerfile
- Build image: docker build -t myrepo/algo:dev
- Test container locally
Create descriptor
- Define parameters and resource requirements
- Specify parameter space if optimizing
- Validate descriptor syntax
Test in Datafye
- Push image to registry
- Provision Foundry environment
- Run backtests
- Iterate on parameters and code

Optimization Workflow

Define parameter space in descriptor
Run genetic algorithm optimization
Review results and scorecards
Select optimal parameters
Update descriptor with optimal values
Validate with out-of-sample backtest

Production Deployment

Finalize parameters based on optimization
Tag stable image version
Update descriptor with production config
Provision Trading environment
Paper trade to validate
Transition to live trading

Best Practices

Descriptor Organization

Keep separate descriptors for different stages:

Development (algo-dev.yaml):

algo:
  name: "momentum-dev"
  image: "myrepo/momentum:dev"
  parameters:
    lookback_period: 20
  resources:
    cpu: 1
    memory: "2Gi"

Optimization (algo-optimize.yaml):

algo:
  name: "momentum-optimize"
  image: "myrepo/momentum:v1.0"
  parameter_space:
    lookback_period: [10, 50]
    threshold: [0.01, 0.05]
  resources:
    cpu: 2
    memory: "4Gi"

Production (algo-prod.yaml):

algo:
  name: "momentum-prod"
  image: "myrepo/momentum:v1.0"
  parameters:
    lookback_period: 28  # Optimized value
    threshold: 0.032     # Optimized value
  resources:
    cpu: 2
    memory: "4Gi"

Version Control

Store descriptors with algo source code
Tag descriptor versions with algo releases
Document parameter changes in commit messages
Use meaningful branch names for experiments

Documentation

Document your algo descriptor:

algo:
  name: "momentum-strategy"
  description: |
    Momentum-based strategy that buys on upward momentum
    and sells on downward momentum. Uses exponential moving
    average with configurable lookback period.

  image: "myrepo/momentum:v1.0"

  parameters:
    lookback_period: 20  # Days to look back for momentum calculation
    threshold: 0.02      # Minimum momentum to trigger trade (2%)

Testing

Before production:

Test with minimal resources first
Validate parameter ranges make sense
Run backtests with various parameter values
Verify outputs match expectations
Check resource usage matches allocation

Next Steps

Learn about optimization — Genetic Algorithm Based Optimization
Understand backtesting — Backtesting Your Algo
See complete schema — Algo Descriptor Reference
Build your first algo — Building Your First Algo - Using Datafye Container

Last updated: 2025-10-11

PreviousThe Algo Container NextScorecarding

Last updated 3 months ago

hashtagPurpose

hashtagWhen You Need Algo Descriptors

hashtagStructure

hashtagBasic Configuration

hashtagAlgo Name

hashtagContainer Image

hashtagParameters

hashtagParameter Space (For Optimization)

hashtagRange Specification

hashtagOptimization Strategy

hashtagResource Requirements

hashtagCPU Allocation

hashtagMemory Allocation

hashtagGPU Support

hashtagInputs and Outputs

hashtagInputs

hashtagOutputs

hashtagExample Algo Descriptors

hashtagExample 1: Simple Momentum Strategy

hashtagExample 2: Optimizable Strategy

hashtagExample 3: ML-Based Strategy

hashtagExample 4: Composable Strategy

hashtagSDK Integration

hashtagParameter Access

hashtagResource Utilization

hashtagInput/Output Handling

hashtagValidation

hashtagSchema Validation

hashtagImage Validation

hashtagParameter Validation

hashtagResource Validation

hashtagVersioning and Updates

hashtagAlgo Versioning

hashtagUpdating Algos

hashtagDevelopment Workflow

hashtagLocal Development

hashtagOptimization Workflow

hashtagProduction Deployment

hashtagBest Practices

hashtagDescriptor Organization

hashtagVersion Control

hashtagDocumentation

hashtagTesting

hashtagNext Steps

Purpose

When You Need Algo Descriptors

Structure

Basic Configuration

Algo Name

Container Image

Parameters

Parameter Space (For Optimization)

Range Specification

Optimization Strategy

Resource Requirements

CPU Allocation

Memory Allocation

GPU Support

Inputs and Outputs

Inputs

Outputs

Example Algo Descriptors

Example 1: Simple Momentum Strategy

Example 2: Optimizable Strategy

Example 3: ML-Based Strategy

Example 4: Composable Strategy

SDK Integration

Parameter Access

Resource Utilization

Input/Output Handling

Validation

Schema Validation

Image Validation

Parameter Validation

Resource Validation

Versioning and Updates

Algo Versioning

Updating Algos

Development Workflow

Local Development

Optimization Workflow

Production Deployment

Best Practices

Descriptor Organization

Version Control

Documentation

Testing

Next Steps