Import

Plugin parameters may be specified as key-value pairs in the --trigger-arguments flag (CLI), in the trigger_arguments field (API) when creating a trigger or via body of HTTP request. This plugin supports TOML configuration files, which can be specified using the config_file_path parameter.

Plugin metadata

This plugin includes a JSON metadata schema in its docstring that defines supported trigger types and configuration parameters. This metadata enables the InfluxDB 3 Explorer UI to display and configure the plugin.

Required parameters

Authentication parameters (required - choose one method)

Method 1: Token-based authentication (InfluxDB v2 or v1 with token support)

Method 2: Username/Password authentication (InfluxDB v1)

Note: You must provide EITHER source_token OR (source_username AND source_password together). Using both methods simultaneously will result in an error.

Optional parameters

TOML configuration

To use a TOML configuration file, set the PLUGIN_DIR environment variable and specify the config_file_path in the trigger arguments. This is in addition to the --plugin-dir flag when starting InfluxDB 3.

Example TOML configuration

https://github.com/influxdata/influxdb3_plugins/blob/main/influxdata/import/import_config.toml

For more information on using TOML configuration files, see the Using TOML Configuration Files section in the influxdb3_plugins/README.md.

Example 1: Basic import with token authentication

Import all data from an InfluxDB v1 instance:

# Create and enable HTTP trigger
influxdb3 create trigger \
  --database mydb \
  --plugin-filename import.py \
  --trigger-spec "request:import" \
  import_trigger

influxdb3 enable trigger --database mydb import_trigger

# Start import via HTTP
curl -X POST http://localhost:8181/api/v3/engine/import?action=start \
  -H "Content-Type: application/json" \
  -d '{
    "source_url": "http://localhost:8086",
    "source_token": "my-super-secret-token",
    "influxdb_version": 1,
    "source_database": "telegraf",
    "dest_database": "imported_data"
  }'

Expected results

• Plugin connects to source InfluxDB at http://localhost:8086 (port from URL)
• Discovers all measurements in the telegraf database
• Estimates import time based on data sampling
• Imports all data to InfluxDB 3 in the imported_data database
• Logs import_id for tracking statistics

Example 2: Time-range import with table filtering

# Start import with time range and table filter
curl -X POST http://localhost:8181/api/v3/engine/import?action=start \
  -H "Content-Type: application/json" \
  -d '{
    "source_url": "http://influxdb-source.example.com:8086",
    "source_username": "admin",
    "source_password": "my-password",
    "influxdb_version": 1,
    "source_database": "telegraf",
    "dest_database": "production_metrics",
    "start_timestamp": "2024-01-01T00:00:00Z",
    "end_timestamp": "2024-12-31T23:59:59Z",
    "table_filter": "cpu.mem.disk.network",
    "import_direction": "newest_first",
    "target_batch_size": 5000
  }'

Expected results

• Imports only cpu, mem, disk, and network measurements
• Processes data from January 1, 2024 to December 31, 2024
• Imports newest data first
• Uses larger batch size (5000 rows) for better performance

Example 3: Pause, check status, and resume import

# Start import (logs import_id, does not return it immediately)
curl -X POST http://localhost:8181/api/v3/engine/import?action=start \
  -H "Content-Type: application/json" \
  -d '{
    "source_url": "http://localhost:8086",
    "source_token": "my-token",
    "influxdb_version": 2,
    "source_database": "large_database",
    "dest_database": "imported"
  }'

# Find import_id from logs:
influxdb3 query --database _internal "SELECT log_text FROM system.processing_engine_logs WHERE trigger_name = 'import_trigger' AND log_text LIKE '%Starting import%' ORDER BY event_time DESC LIMIT 1"

# Set the import_id from logs
IMPORT_ID=""

# Pause import (e.g., during high-traffic hours)
curl -X POST "http://localhost:8181/api/v3/engine/import?action=pause&import_id=$IMPORT_ID"

# Check status after import completion (paused, cancelled, or completed)
curl "http://localhost:8181/api/v3/engine/import?action=status&import_id=$IMPORT_ID"

# Resume later
curl -X POST "http://localhost:8181/api/v3/engine/import?action=resume&import_id=$IMPORT_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "source_token": "my-token"
  }'
</code></pre>

### Expected results

- Import starts and logs a unique import_id (check logs to obtain it)
- Import continues running in the background, logging progress
- Pause command stops import gracefully at current position
- Status endpoint returns comprehensive statistics **only after import completion** (paused, cancelled, or finished)
- Resume command continues from the exact point where it was paused and returns final results upon completion

### Example 4: Dry run for import plan

curl -X POST http://localhost:8181/api/v3/engine/import?action=start \
  -H "Content-Type: application/json" \
  -d '{
    "source_url": "http://localhost:8086",
    "source_token": "my-token",
    "influxdb_version": 1,
    "source_database": "telegraf",
    "dry_run": true
  }'


### Expected results

With dry_run: true, the plugin generates a comprehensive import plan **without processing any data**. It only performs:
- Schema inspection (tags and fields)
- Data sampling for time estimation
- Conflict detection

The response returns immediately with a detailed import plan:

{
  "import_id": "abc123...",
  "status": "dry_run_plan",
  "source": {
    "url": "http://localhost:8086",
    "database": "telegraf",
    "influxdb_version": 1
  },
  "destination": {
    "database": "imported_data"
  },
  "time_range": {
    "start": "all data",
    "end": "all data"
  },
  "import_settings": {
    "direction": "oldest_first",
    "target_batch_size": 2000,
    "query_interval_ms": 100
  },
  "tables": {
    "total": 5,
    "list": ["cpu", "mem", "disk", "network", "processes"],
    "filtered": "all tables"
  },
  "estimated_import": {
    "total_rows": 5000000,
    "estimated_duration": "1 hour 15 minutes",
    "estimated_duration_seconds": 4500,
    "per_table_estimates": [
      {
        "measurement": "cpu",
        "estimated_rows": 1000000,
        "estimated_seconds": 900
      },
      {
        "measurement": "mem",
        "estimated_rows": 800000,
        "estimated_seconds": 720
      }
    ]
  },
  "schema_conflicts": {
    "total": 2,
    "details": [
      {
        "measurement": "cpu",
        "type": "tag_field_conflict",
        "conflicts": ["host", "region"],
        "resolution": "Tags will be renamed with '_tag' suffix: host -> host_tag, region -> region_tag"
      }
    ]
  }
}


**Note**: Dry run mode is fast and lightweight - it does not query or process any actual data points, only metadata. Use it to:
- Preview import scope and estimates
- Identify schema conflicts before import
- Validate configuration and connectivity
- Plan import time windows