Salesforce and HubSpot

A dataset can leave Studio as the records a CRM holds: Salesforce's standard objects or HubSpot's default objects, with every link between them kept. Each export is a zip of CSVs in load order, a manifest, a README and a loader script, so the data goes into a real org or portal in one command.

Getting the export#

• Studio: open a dataset, choose Export, and pick Salesforce or HubSpot under CRM. Both are offered on the relational shape.
• API: POST /v1/datasets/{id}/exports with {"format": "salesforce"} or {"format": "hubspot"}. See The API.
• MCP: export_dataset with format set to salesforce or hubspot.

Which tables become which records#

Studio reads the dataset's structure, not its table names alone:

A business that sells to people, with no table of companies, loads its customers as contacts. Tables no standard object holds (subscriptions, invoices, usage) are left out, and the README names each one. Every column with no standard home is carried whole as a custom field or property, created before the load.

The built-in SaaS and sales shape fills every object: customers and prospects, their people, won, lost and open deals, support tickets and the sales team's calls, emails and meetings.

Salesforce#

What is in the zip#

Objects and load order#

User, Account, Contact, Opportunity, Case, then Task and Event. A record's parents always load before it.

Every record carries the dataset's own key in Misata_Id__c, an external id. Every lookup names its parent by that key: a contact's account is the column Account.Misata_Id__c, and an activity on a deal is Opportunity:What.Misata_Id__c. The files upsert with Bulk API 2.0 in order, so each lookup resolves to a record loaded earlier, and a second load updates the same records instead of adding new ones.

Values a new org accepts#

• Picklists use the values a new org ships with: stages, account types, industries, lead sources, case statuses, priorities and origins, task and event types. A column fills a standard picklist only when every one of its values has a place there. Otherwise it is kept as a custom field of its own, so nothing it says is lost.
• Opportunity stages. The dataset's stages are placed on a new org's stages in order: its won stage on Closed Won, its lost stage on Closed Lost. Probability and forecast category follow the stage, as a new org sets them.
• Account type is Customer - Direct for a company with a contract, and Prospect for one without.
• Activities. A meeting with a duration loads as an Event; every other activity as a Task, Completed when it is dated on or before the data's last day and Not Started after it. Each sits on its opportunity, else on its account.
• Required fields the dataset does not state are worked out and listed in the manifest. An opportunity's name is its account's name and the month the deal opened. An open deal's close date is its creation plus the median time the dataset's closed deals took.
• Dates are YYYY-MM-DD and times UTC.

Loading#

The loader needs Python 3.9 or later and requests. It reads the org from the environment, never prints it, and runs from the unzipped folder:

pip install requests
SF_INSTANCE_URL=https://<my-domain>.my.salesforce.com SF_ACCESS_TOKEN=... \
  python load_salesforce.py --create-fields

For an External Client App with the client credentials flow, set SF_DOMAIN_URL, SF_CLIENT_ID and SF_CLIENT_SECRET instead.

Before anything loads, the loader reads the org. With --create-fields it creates the custom fields through the Tooling API and gives the loading user access to them in a permission set named Misata_Load; without it, it lists any field still missing and stops. It checks every picklist value against the org's own: a value the org lacks stops the load, or is left empty with --blank-invalid. Then it upserts each file in order. A job with failed rows stops the load, and its rows are saved under failed/ with Salesforce's error on each.

Owners. 01_User.csv lists the people who own records. By default (--owners match) each is matched to an active user with the same email, and a record whose owner has no user is owned by the loading user. --owners create creates the users (a licence each; --notify-email names the one address their welcome emails go to, and --profile their profile). --owners me gives everything to the loading user.

What a load cannot set#

• Stage and field history are written by Salesforce as records change. The time a deal reached each stage is carried in its own fields instead.
• Created dates are set on accounts, contacts and opportunities only when the org has "Set Audit Fields upon Record Creation" turned on for the loading user. The loader checks, and the date is kept in its own field either way.
• Storage. Most records count 2 KB against data storage. The README and manifest give the export's total; a Developer Edition org holds 5 MB. For more, build with fewer accounts or load into a sandbox or trial org.

To use Data Loader or another tool instead, upsert each file in order on Misata_Id__c.

HubSpot#

What is in the zip#

Objects and load order#

Companies, contacts, deals, tickets, then calls, emails, meetings and tasks.

Every record carries the dataset's own key in the property misata_id, unique on the objects that upsert on it. Each association:<object> column names the record it belongs to by that key. Associations use HubSpot's own types: a contact, deal or ticket gets its company as its primary company, and an activity is associated with the company and the deal it names.

Values a new portal accepts#

• Deals go to the default sales pipeline: Appointment Scheduled, Qualified To Buy, Presentation Scheduled, Decision Maker Bought-In, Contract Sent, Closed Won, Closed Lost. The dataset's stages are placed on them in order.
• Tickets go to the default support pipeline: New, Waiting on contact, Waiting on us, Closed. Priorities are LOW, MEDIUM and HIGH.
• Lifecycle stage. A company with a contract is a customer, and one without is an opportunity.
• Calls are outbound, completed when dated on or before the data's last day and queued after it. Where the dataset records them, each has its duration and one of HubSpot's default call outcomes (connected, no answer, left voicemail and so on).
• Emails are sent, or bounced or failed where the dataset says so. Meetings are completed when past and scheduled when ahead, or no-show, cancelled or rescheduled where the dataset says so. Any other kind of activity is a task.

Loading#

The loader needs Python 3.9 or later and requests. Run it from the unzipped folder with a private app's access token that has the CRM object, schema and owner scopes:

pip install requests
HUBSPOT_TOKEN=... python load_hubspot.py --create-properties

With --create-properties it creates the properties the files need; without it, it lists any that are missing and stops. It then reads the portal's pipelines, owners and property options. Stages are matched by id, else by label, and the load stops on a stage it cannot place. Dropdown values such as industry and lifecycle stage are matched to the portal's options by label. Owners are matched by email; a person with no user in the portal leaves the record unassigned and is listed. Then each file is upserted on misata_id in order, and each record is associated with the records its association: columns name.

Activities are created once. The ids HubSpot gave them are kept in hubspot_ids.json in the folder, so a second run adds nothing twice. A batch with errors stops the load, and its rows are saved under failed/.

The import requests in imports/ set properties only. Owners and associations need the loader.

What a load cannot set#

• Stage history. The time a deal entered each stage is worked out by HubSpot as a deal moves. Each stage's time is carried in its own property instead.
• Create dates of contacts and companies are set by HubSpot when they load. The dataset's dates are kept in their own properties. A deal's and a ticket's create date is set from the data.

Load from the API#

Misata can run either loader itself, so your product loads a portal or an org with one call and nothing to install. The same files go in the same order with the same checks as above.

# plan: each object and how many records, and with a token, the properties it would create first
curl -X POST https://api.misata.studio/v1/datasets/$ID/load -H "Authorization: Bearer $MISATA_KEY" \
  -H "Content-Type: application/json" -d '{"target": "hubspot", "token": "'$HUBSPOT_TOKEN'"}'

# load
curl -X POST https://api.misata.studio/v1/datasets/$ID/load -H "Authorization: Bearer $MISATA_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target": "salesforce", "token": "'$SF_ACCESS_TOKEN'", "instance_url": "https://acme.my.salesforce.com", "apply": true}'

A load answers with each object and how many records it wrote, and the loader's own lines. Records are upserted on the Misata id, so loading again updates the same records. HubSpot activities are created once per portal: a second load adds none again. Salesforce owners are matched to active users by email, and a record whose owner has no user is owned by the loading user.

The token is used for that one call and is never stored or logged. Misata reaches only api.hubapi.com or your org's own *.salesforce.com address, over HTTPS, and follows no redirect. A refused token, a missing scope or a picklist value the org lacks comes back as a 422 naming the field to change. One call loads up to 100,000 records; past that, take the export and run its loader where you are.

Bring an org's own shape#

To build data for an org or portal that already has its own fields, import its structure instead of describing it. On New dataset, choose Import a structure and add:

• Salesforce: an object's describe JSON (sobjects/<Object>/describe, or describeSObjects for several), custom objects included. Each object becomes a table, picklists keep their active values, and lookups become links between the objects you include. Fields Salesforce keeps for itself (audit and calculated fields) are left out, and said.
• HubSpot: an object's properties (GET /crm/v3/properties/<object>, the file named for the object, such as deals.json) or object schemas (GET /crm/v3/schemas), custom objects included. Dropdowns keep their options, and each record's primary association becomes a link. Calculated and read-only properties are left out, and counted.

Then state the numbers on those tables as with any imported structure.