Performance Issues

This guide helps diagnose and resolve performance-related problems with arb-assist.

CPU Performance

High CPU Usage

Symptoms:

Consistent 100% CPU usage
Slow response times
Delayed config updates
System becomes unresponsive

Diagnosis:

# Check CPU usage
htop
# or
top -p $(pgrep arb-assist)

# Check process details
ps aux | grep arb-assist

CPU Optimization Strategies

1. Reduce Processing Frequency

Increase update intervals:

update_interval = 30000  # 30 seconds (from 10)
run_interval = 120000    # 2 minutes (from 1)

2. Limit Data Scope

Reduce the amount of data processed:

# Fewer mints to track
mints_to_rank = 10  # From 50

# Fewer pools per mint
filter_thresholds = [{
  top_pool_num = 2,  # From 5
}]

# Filter programs
filter_programs = true
arb_programs = ["MEVi...", "NA24..."]  # Only proven programs

3. Simplify Computations

Use simpler sorting strategies:

# Simple metric
intermint_sort_strategy = { metric = "profit", direction = "descending" }

# Avoid complex metrics like volatility or turnover

4. Enable Caching

Reduce redundant calculations:

# Longer halflife = less frequent recalculation
halflife = 300000  # 5 minutes

CPU Profiling

Monitor which operations consume most CPU:

Enable logging:

log_output = true

Check operation timings in logs:

[DEBUG] GRPC processing: 234ms
[DEBUG] Mint ranking: 89ms
[DEBUG] Config generation: 45ms

Identify bottlenecks and optimize accordingly

Memory Performance

Memory Leaks

Symptoms:

Gradually increasing memory usage
Eventually crashes with OOM
System swap usage increases

Diagnosis:

# Monitor memory usage over time
watch -n 5 'ps aux | grep arb-assist | grep -v grep'

# Check system memory
free -h

# Monitor swap usage
vmstat 1

Memory Optimization

1. Reduce Data Retention

Faster decay of old data:

halflife = 60000  # 1 minute (aggressive decay)

2. Limit Tracked Entities

Reduce memory footprint:

mints_to_rank = 15      # Lower limit
aluts_per_pool = 5      # Fewer ALUTs
dexes = [               # Only essential DEXes
  "675k...",  # Raydium
  "whir...",  # Orca
]

3. Configure Garbage Collection

For systems with limited memory:

# Restart periodically to clear memory
# Add to crontab:
0 */4 * * * systemctl restart arb-assist

Memory Usage Estimation

Calculate approximate memory usage:

Base overhead: ~50 MB
Per mint: ~100 KB
Per pool: ~10 KB
Per ALUT: ~1 KB

Total = 50 MB + (mints * 100 KB) + (mints * pools * 10 KB) + (mints * aluts * 1 KB)

Example with 50 mints, 5 pools each, 20 ALUTs:

50 MB + (50 * 100 KB) + (50 * 5 * 10 KB) + (50 * 20 * 1 KB)
= 50 MB + 5 MB + 2.5 MB + 1 MB
= 58.5 MB

Network Performance

Bandwidth Issues

Symptoms:

GRPC disconnections
Incomplete data
Slow updates
RPC timeouts

Diagnosis:

# Monitor network usage
iftop
# or
nethogs

# Check bandwidth
speedtest-cli

# Monitor specific connection
tcpdump -i any host grpc.example.com

Network Optimization

1. GRPC Stream Filtering

Reduce data volume by filtering at source:

filter_programs = true  # Only specific programs
exclude_mints = [       # Skip unnecessary tokens
  "So11111111111111111111111111111111111112",
  "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
]

2. RPC Optimization

Distribute load across multiple endpoints:

# Different RPCs for different purposes
rpc_url = "https://data-rpc.com"      # For reads
config_rpc_url = "https://fast-rpc.com" # For bot
sending_rpc_urls = [                    # For sends
  "https://send1.com",
  "https://send2.com",
]

3. Connection Pooling

For NotArb, use dedicated connections:

[notarb]
token_accounts_checker = { rpc_url = "https://slow-rpc.com", delay_seconds = 5 }
blockhash_updater = { rpc_url = "https://fast-rpc.com", delay_ms = 300 }

Latency Optimization

Reduce network latency:

Geographic Proximity:
- Use RPC/GRPC endpoints near your server
- Consider multi-region deployment
Connection Reuse:
- Maintain persistent connections
- Avoid connection churn
Batch Operations:
- Group related requests
- Reduce round trips

Disk I/O Performance

Slow Config Updates

Symptoms:

Delays in writing config files
File system errors
Disk space issues

Diagnosis:

# Check disk usage
df -h

# Monitor disk I/O
iostat -x 1

# Check write speed
dd if=/dev/zero of=test.file bs=1M count=1000

Disk Optimization

1. Use Fast Storage

Prefer NVMe SSD over HDD
Use local storage over network storage
Ensure adequate free space (>20%)

2. Reduce Write Frequency

For less critical updates:

update_interval = 60000  # Update every minute

3. Optimize File Operations

Use efficient file writing:

# Disable unnecessary outputs
log_output = false

# Use single output mode
mode = "smb"  # Not "both"

System Resource Monitoring

Comprehensive Monitoring Setup

Install monitoring tools:

sudo apt install htop iotop iftop sysstat

Create monitoring script:

#!/bin/bash
# monitor-arb-assist.sh
while true; do
  echo "=== $(date) ==="
  ps aux | grep arb-assist | grep -v grep
  free -h | grep -E "Mem|Swap"
  iostat -x 1 1 | grep -E "avg-cpu|sda"
  echo ""
  sleep 60
done

Log resource usage:

./monitor-arb-assist.sh > resource-usage.log 2>&1 &

Performance Baselines

Typical resource usage for reference:

Metric

Light Load

Medium Load

Heavy Load

CPU

10-20%

30-50%

70-90%

Memory

100-200 MB

300-500 MB

1-2 GB

Network

10-50 Mbps

50-100 Mbps

100-200 Mbps

Disk I/O

<1 MB/s

1-5 MB/s

5-10 MB/s

Performance Tuning Workflow

Step 1: Identify Bottleneck

Check CPU usage
Monitor memory consumption
Analyze network traffic
Review disk I/O

Step 2: Apply Optimization

Based on bottleneck:

CPU: Reduce processing frequency/scope
Memory: Limit data retention/tracking
Network: Filter streams, use better endpoints
Disk: Use faster storage, reduce writes

Step 3: Measure Impact

Record baseline metrics
Apply single change
Monitor for 1-2 hours
Compare metrics
Keep or revert change

Step 4: Iterate

Continue optimizing until:

Performance is acceptable
Resource usage is sustainable
Arbitrage effectiveness maintained

Advanced Performance Tips

1. Process Priority

Increase arb-assist priority:

# Nice value (-20 highest, 19 lowest)
sudo renice -n -10 -p $(pgrep arb-assist)

2. CPU Affinity

Dedicate CPU cores:

# Assign to cores 2 and 3
taskset -cp 2,3 $(pgrep arb-assist)

4. File System Optimization

Use appropriate mount options:

# For SSD in /etc/fstab
/dev/sda1 / ext4 defaults,noatime,nodiratime 0 1

When to Scale

Consider scaling when:

Single instance can't handle load
Need geographic distribution
Require redundancy
Want to specialize strategies

Scaling options:

Vertical: Upgrade server specs
Horizontal: Multiple instances
Distributed: Geographic spread
Specialized: Different configs for different strategies

PreviousCommon Errors NextConnection Problems

Last updated 6 months ago

hashtagCPU Performance

hashtagHigh CPU Usage

hashtagCPU Optimization Strategies

hashtag1. Reduce Processing Frequency

hashtag2. Limit Data Scope

hashtag3. Simplify Computations

hashtag4. Enable Caching

hashtagCPU Profiling

hashtagMemory Performance

hashtagMemory Leaks

hashtagMemory Optimization

hashtag1. Reduce Data Retention

hashtag2. Limit Tracked Entities

hashtag3. Configure Garbage Collection

hashtagMemory Usage Estimation

hashtagNetwork Performance

hashtagBandwidth Issues

hashtagNetwork Optimization

hashtag1. GRPC Stream Filtering

hashtag2. RPC Optimization

hashtag3. Connection Pooling

hashtagLatency Optimization

hashtagDisk I/O Performance

hashtagSlow Config Updates

hashtagDisk Optimization

hashtag1. Use Fast Storage

hashtag2. Reduce Write Frequency

hashtag3. Optimize File Operations

hashtagSystem Resource Monitoring

hashtagComprehensive Monitoring Setup

hashtagPerformance Baselines

hashtagPerformance Tuning Workflow

hashtagStep 1: Identify Bottleneck

hashtagStep 2: Apply Optimization

hashtagStep 3: Measure Impact

hashtagStep 4: Iterate

hashtagAdvanced Performance Tips

hashtag1. Process Priority

hashtag2. CPU Affinity

hashtag4. File System Optimization

hashtagWhen to Scale

CPU Performance

High CPU Usage

CPU Optimization Strategies

1. Reduce Processing Frequency

2. Limit Data Scope

3. Simplify Computations

4. Enable Caching

CPU Profiling

Memory Performance

Memory Leaks

Memory Optimization

1. Reduce Data Retention

2. Limit Tracked Entities

3. Configure Garbage Collection

Memory Usage Estimation

Network Performance

Bandwidth Issues

Network Optimization

1. GRPC Stream Filtering

2. RPC Optimization

3. Connection Pooling

Latency Optimization

Disk I/O Performance

Slow Config Updates

Disk Optimization

1. Use Fast Storage

2. Reduce Write Frequency

3. Optimize File Operations

System Resource Monitoring

Comprehensive Monitoring Setup

Performance Baselines

Performance Tuning Workflow

Step 1: Identify Bottleneck

Step 2: Apply Optimization

Step 3: Measure Impact

Step 4: Iterate

Advanced Performance Tips

1. Process Priority

2. CPU Affinity

4. File System Optimization

When to Scale