provision

This guide shows you how to provision a Datafye Foundry deployment in Amazon Web Services.

Prerequisites

Before you begin, ensure:

• The Datafye CLI is installed (see Installation)

• AWS credentials are configured (see AWS Setup)

• You have created the required descriptor files (see Deployment Descriptors)

Provision Command

Use the following command to provision an AWS Foundry deployment:

datafye foundry aws provision [options]

Parameters

Required Parameters

Parameter	Description
--name <name>	Deployment name (must be unique per AWS profile)
--profile <profile>	AWS CLI profile to use (configured in AWS setup)

Descriptor Parameters

You must provide descriptor files using one of the following approaches:

Option 1: Combined descriptor file

Parameter	Description
--descriptor <path>	Path to a combined descriptor file containing data (and optionally algo) configuration

Option 2: Separate descriptor files

Parameter	Description
--data-descriptor <path>	Path to data descriptor file (required)
--algo-descriptor <path>	Path to algo descriptor file (optional - only for Full Stack scenario)

Optional Parameters

Parameter	Description	Default
--region <region>	AWS region to provision in	us-east-1
--cidr <cidr>	CIDR block for VPC (format: x.y.z.w/nn)	10.0.0.0/16

Run datafye help foundry aws provision to see the complete parameter list and usage.

Foundry Scenarios

AWS Foundry deployments support two scenarios:

Data Cloud Only

Provisions only the Data Cloud, allowing you to use your own algo containers.

Required descriptors:

• Data descriptor only

Example:

datafye foundry aws provision \
  --name my-data-foundry \
  --profile datafye \
  --data-descriptor data.yaml

Full Stack

Provisions Data Cloud, Algo Container runtime, Backtesting Engine, and MCP Server.

Required descriptors:

• Data descriptor

• Algo descriptor

Example:

datafye foundry aws provision \
  --name my-full-foundry \
  --profile datafye \
  --data-descriptor data.yaml \
  --algo-descriptor algo.yaml \
  --region us-west-2 \
  --cidr 10.1.0.0/16

Or using a combined descriptor:

datafye foundry aws provision \
  --name my-foundry \
  --profile datafye \
  --descriptor foundry.yaml

Provisioning Process

During provisioning, the CLI will:

1. Validate descriptors - Check schema, semantics, and dependencies

2. Create VPC and networking - Provision Virtual Private Cloud, subnets, security groups

3. Provision EC2 instances - Create virtual machines for Datafye services

4. Deploy services - Install Data Cloud (and Algo Container/Backtesting/MCP if Full Stack)

5. Configure connectivity - Connect to data providers using your credentials

6. Run health checks - Verify all services are running correctly

7. Display access information - Show URLs and connection details

AWS provisioning typically takes 10-20 minutes. Do not interrupt the provisioning process.

After Provisioning

Once provisioning completes successfully, the CLI will display:

• Deployment ID and status

• API base URLs for accessing Datafye services

• WebSocket endpoint for real-time data streams

• MCP server endpoint (Full Stack only)

• SSH access information (if needed for troubleshooting)

Accessing Your Deployment

The AWS deployment will be accessible via the URLs provided in the output:

• Data Cloud REST API: http://<deployment-endpoint>/datafye-api/v1/

• Data Cloud WebSocket: ws://<deployment-endpoint>/datafye-ws

• MCP Server (Full Stack only): http://<mcp-endpoint>:3000

Validate Connectivity

Test your deployment with:

# Check Data Cloud health (replace with your deployment endpoint)
curl http://<deployment-endpoint>/datafye-api/v1/health

# Get security master
curl http://<deployment-endpoint>/datafye-api/v1/stocks/reference/security-master

Your AWS Foundry deployment is now running and ready to use!

Next Steps

After provisioning:

• Data Cloud Only: Connect your own algo containers to the Data Cloud APIs (see Building Your First Algo - Using Own Container)

• Full Stack: Start developing your algo using the SDK (see Building Your First Algo - Using Datafye Container)

• Run backtests: Test your strategy with historical data (see Backtesting Your Algo)

Troubleshooting

AWS Credential Errors

If you see authentication errors:

1. Verify your AWS profile is configured: aws sts get-caller-identity --profile datafye

2. Ensure your credentials have the necessary permissions (see AWS Setup)

3. Check that you're using the correct profile name

Deployment Name Conflicts

If you receive an error about the deployment name already existing:

1. Choose a different deployment name with --name

2. Or deprovision the existing deployment: datafye foundry aws deprovision --name <existing-name> --profile <profile>

3. Retry the provision command

Resource Limit Errors

If you hit AWS resource limits:

1. Check your AWS account limits in the EC2 dashboard

2. Request limit increases if needed through AWS Support

3. Or deprovision unused deployments to free resources

VPC CIDR Conflicts

If you encounter VPC address conflicts:

1. Try a different CIDR block with --cidr (e.g., 10.1.0.0/16, 10.2.0.0/16)

2. Ensure the CIDR doesn't conflict with existing VPCs in your account

3. Check existing VPCs: aws ec2 describe-vpcs --profile <profile>

Region Unavailability

If services are not available in your chosen region:

1. Verify the region supports required AWS services (EC2, VPC, Route53)

2. Try a different region with --region (e.g., us-east-1, us-west-2)

3. Contact Datafye support if you require a specific region

Descriptor Validation Errors

If descriptor validation fails:

1. Review the error message for specific field issues

2. Check descriptor format matches the schema (see Deployment Descriptors)

3. Verify referenced resources exist (container images, symbols)

4. Ensure credentials are provided via environment variables if using ${VAR} syntax

Data Provider Connection Errors

If data provider connections fail:

1. Verify your data provider credentials are correct

2. Check environment variables are set if using variable substitution

3. Ensure your data provider account is active and has necessary permissions

4. Review data provider status (some providers may have API outages)

Related Commands

• Start an AWS Foundry - Start a stopped deployment

• Stop an AWS Foundry - Stop a running deployment

• Deprovision an AWS Foundry - Remove the deployment

---

Last updated: 2025-10-22