--- title: "Load Historical Data into Insider One" slug: "load-historical-data-into-insider-one" description: "Learn how to load historical profiles and events into Insider One. Choose a method, check its limits, follow the right load order, and plan event retention." updated: 2026-11-19T09:23:31Z published: 2026-11-19T09:23:31Z canonical: "academy.insiderone.com/load-historical-data-into-insider-one" --- > ## Documentation Index > Fetch the complete documentation index at: https://academy.insiderone.com/llms.txt > Use this file to discover all available pages before exploring further. # Load Historical Data into Insider One Your customers' history, such as past purchases, profile attributes, and consent records, helps you segment, personalize, and run campaigns from day one. You usually load it once, during implementation or when you move from a previous tool, and then keep it current with regular delta loads. This guide helps you choose a method to load your data, check its limits, load your data in the right order, and plan for event retention. In this article, you will learn how to: - [Choose a method and check its limits](/docs/load-historical-data-into-insider-one#choose-a-method-and-check-its-limits) - [Load your data in the right order](/docs/load-historical-data-into-insider-one#load-your-data-in-the-right-order) - [Plan for event retention](/docs/load-historical-data-into-insider-one#plan-for-event-retention) - [Plan a load of 6 million profiles](/docs/load-historical-data-into-insider-one#example-plan-a-load-of-6-million-profiles) - [Move data from a previous tool](/docs/load-historical-data-into-insider-one#move-data-from-a-previous-tool) ## Choose a method and check its limits Start with where your history lives today: in a data warehouse, in files exported from another system, or in a system your team can connect to the Insider One application programming interface (API). Then compare your expected volume with the limits of that method. | Method | Best for | Limits | Guide | | --- | --- | --- | --- | | Panel upload in Static Segments | Lists of profiles and attributes that you upload as a comma-separated values (CSV) file in the InOne panel | Upload a CSV file no larger than 100 MB or a XLS/XLSX file no larger than 5 MB. Make sure that your file contains up to 2 million lines of data. Upload time varies with the number of rows, columns, and file size. | [Static Segments](https://academy.insiderone.com/docs/static-segments) | | File Transfer Protocol (FTP) or Secure File Transfer Protocol (SFTP) | CSV or tab-separated values (TSV) files that your team exports from another system to a file server | Up to 2 gigabytes (GB) per file. For the best performance, keep files under 500 megabytes (MB). | [Import Data from FTP/SFTP to InOne](https://academy.insiderone.com/docs/import-data-from-ftp-and-sftp-to-inone) | | Amazon Simple Storage Service (Amazon S3) | Large batch files, when your data pipeline already runs on Amazon Web Services (AWS) | 5 GB per file, 25 files per day, 20 GB per day in total, and 5 MB per row | [Import Data from Amazon S3](https://academy.insiderone.com/docs/import-data-from-amazon-s3) | | Upsert User Data API | Profiles and events that your backend or integration job sends in batches | 5 MB and 1,000 users per request. 25,000 requests per minute, shared with the Delete User Attribute API, which allows up to 25 million user records per minute with full batching. | [Upsert User Data](https://academy.insiderone.com/docs/upsert-user-data-api) | | Snowflake or Google BigQuery sync | History that already sits in your data warehouse | 1 billion rows per sync, including the initial full sync, and 350 kilobytes (KB) per row | [Import Data from Snowflake to Insider One](https://academy.insiderone.com/docs/import-data-from-snowflake-to-insider-one), [Send Google BigQuery Data to Insider One](https://academy.insiderone.com/docs/send-google-bigquery-data-to-insider-one) | | Databricks sync | History in Delta tables with Change Data Feed enabled | 1 billion records per sync. The initial sync counts table rows, and later syncs count change records. 350 KB per row. | [Import Data from Databricks to Insider One](https://academy.insiderone.com/docs/import-data-from-databricks-to-insider-one) | | Amazon Redshift | History in Amazon Redshift that you export to your Amazon S3 bucket | The Amazon S3 limits apply, because Insider One imports the exported files from Amazon S3. | [Import Data from Amazon Redshift to Insider One](https://academy.insiderone.com/docs/import-data-from-amazon-redshift-to-insider-one) | The first Snowflake or Google BigQuery sync imports the full table without triggering webhooks, campaigns, or journeys. Later syncs import only new or changed rows. If a sync goes above 1 billion rows, the integration moves to Archived and that sync is not processed. If your sync could get close to this limit, contact the Insider One team before you launch it. > [!NOTE] > These are the standard limits. If your load approaches a limit, or you need help to plan throughput, concurrency, or completion time, contact the Insider One team. For more details, refer to [Data Ingestion Limits and Performance](https://academy.insiderone.com/docs/data-ingestion-limits-and-performance). ## Load your data in the right order Follow these steps to load your history smoothly: 1. **Plan your user data and identifiers.** Complete [User Data Planning](https://academy.insiderone.com/docs/user-data-planning) and review your [Identity Resolution Management Settings](https://academy.insiderone.com/docs/identity-resolution-management-settings) before the first import, so loaded records match the right profiles. 2. **Create your custom attributes, events, and event parameters.** Navigate to **Components** > **Attributes and Events** and create everything you plan to load before you configure a file or warehouse import. 3. **Load identifiers and profile attributes.** This order is recommended, not required. When your profiles are in place first, you can check on the User Profiles page that users are matched and merged as you expect before you load your events, which are usually the larger volume. 4. **Load historical events with their original timestamps.** To keep past activity out of live journeys, use the setting that matches your method: - In the Upsert User Data API, send each event's original time in `timestamp` and set `skip_hook` to `true`. - In an FTP/SFTP integration, set **Timestamp source** to **From column** in the **Event Mapping** step, and enable **Skip Hook** in the **Additional Settings** step. ![](https://cdn.document360.io/c6df4583-da94-4cb2-bb8a-be0cbdd11109/Images/Documentation/image-C147NHRJ.png) ![](https://cdn.document360.io/c6df4583-da94-4cb2-bb8a-be0cbdd11109/Images/Documentation/image-L1WFXH36.png) - Amazon S3 imports have no Skip Hook setting. Insider One suppresses journey and data stream triggers automatically when more than 400,000 rows are parsed for your account within a rolling 15-minute window. Below that volume, loaded events can trigger live journeys. To keep a smaller historical load out of live journeys, use [FTP/SFTP](/v1/docs/import-data-from-ftp-and-sftp-to-inone) with Skip Hook or the [Upsert User Data API](/v1/docs/upsert-user-data-api) with skip_hook. For details, refer to [Automatic journey-trigger throttling](https://academy.insiderone.com/docs/data-ingestion-limits-and-performance#automatic-journeytrigger-throttling). 5. **Check your data.** Review a sample of loaded users on the [User Profiles](https://academy.insiderone.com/docs/user-profiles-overview) page, then start your regular delta loads. > [!NOTE] > Before you send events, review [Integration Pitfalls](https://academy.insiderone.com/docs/integration-pitfalls). For example, a request can return 200 OK even when some records failed validation. ## Plan for event retention Insider One keeps each event for its time to live (TTL). The TTL is counted from the event's own timestamp, not from the day you load it. For example, if you load an event that happened one month ago and its TTL is three months, the event expires two months after you load it. Insider One writes an event to the user profile only when its timestamp falls within the event's TTL. A request can still return 200 OK when an expired event is left out, so remove expired events from your files and payloads before you load them. By default, Purchase events are kept for 730 days, and some events, such as Sign-Up, are kept for 365 days. Custom events are kept for three months, and other events that are not in the retention table are kept for 90 days. For the period of each default event, refer to [Data Retention](https://academy.insiderone.com/docs/data-retention). If you want to keep your loaded events for longer, contact the Insider One team before you load them. The Insider One team can extend Event TTL and User TTL for your account. Profiles also have their own retention period. Insider One removes a user whose data has gone 12 months or more without an update. The Attributes and Events ingestion graph shows when Insider One received data. This means an expired historical event can appear in the graph even though it is not stored on the user profile. ## Example: Plan a load of 6 million profiles This example uses the Upsert User Data API limits. Suppose you need to load 6,000,000 customer profiles with their attributes. 1. **Batch size:** Each request carries up to 1,000 users and up to 5 MB, so the load needs at least 6,000 requests. If 1,000 of your profiles add up to more than 5 MB, use smaller batches. At 500 users per request, the load needs 12,000 requests. 2. **Request rate:** The Upsert User Data API and the Delete User Attribute API share a limit of 25,000 requests per minute. Pace your job so that the load and any other traffic on these endpoints stay within that limit. If a request receives a 429 Too Many Requests response, retry it with exponential backoff. 3. **Processing time:** The rate limit controls how fast you can send requests, which is different from how long processing takes. To plan the processing time for your load, contact the Insider One team. ## Move data from a previous tool Treat a move from another tool as a historical load. These steps help you carry your data over: - Export the identifiers, profile attributes, opt-in and consent attributes, and events you want to keep, either as files or through your previous tool's API. - Map them to your Insider One data plan. If you already have a list of event, attribute, and parameter names, upload it in bulk with [Data Name Upload](https://academy.insiderone.com/docs/data-name-upload). - Decide how user identifiers from the previous tool carry over before you load profiles. For more details, refer to [How Identifiers Work in Insider One](https://academy.insiderone.com/docs/how-identifiers-work-in-insider-one). - Load permissions as opt-in attributes, such as `email_optin`, `sms_optin`, and `whatsapp_optin`, with your import method. For the attribute list, refer to [Upsert User Data](https://academy.insiderone.com/docs/upsert-user-data-api). - If your data is in Salesforce Marketing Cloud Data Extensions, use the guided migrator in [Send Salesforce Marketing Cloud (SFMC) Data to Insider One](https://academy.insiderone.com/docs/send-salesforce-marketing-cloud-sfmc-data-to-insider-one). It transfers identifiers, attributes, preferences, and consent data. - If a user acceptance testing (UAT) account is linked to your production account, test your Upsert User Data API integration there first, and then load your production data. ## Related - [Data Ingestion](/data-ingestion.md) - [Data Ingestion Limits and Performance](/data-ingestion-limits-and-performance.md) - [Data Retention](/data-retention.md) - [Integration Pitfalls](/integration-pitfalls.md) - [Data Warehouse and Storage Import Monitoring and Error Guide](/data-warehouse-and-storage-import-monitoring-error-guide.md)