<!-- Source: https://www.misata.studio/docs/learn/crm -->
<!-- Docs index: https://www.misata.studio/llms.txt -->

# Salesforce and HubSpot

A dataset can leave Studio as the records a CRM holds: Salesforce's standard objects or HubSpot's default objects, with
every link between them kept. Each export is a zip of CSVs in load order, a manifest, a README and a loader script, so
the data goes into a real org or portal in one command.

## Getting the export

- **Studio:** open a dataset, choose **Export**, and pick **Salesforce** or **HubSpot** under CRM. Both are offered on
  the relational shape.
- **API:** `POST /v1/datasets/{id}/exports` with `{"format": "salesforce"}` or `{"format": "hubspot"}`. See
  [The API](/docs/learn/api).
- **MCP:** `export_dataset` with `format` set to `salesforce` or `hubspot`.

## Which tables become which records

Studio reads the dataset's structure, not its table names alone:

| Part of a CRM | The table that plays it |
|---|---|
| Accounts or companies | The table of organisations the most other tables point to |
| Contacts | People with an email who belong to an account. With no such table, one contact per distinct email named on the account and its other tables |
| Opportunities or deals | A table under the account with an amount and both won and lost stages |
| Cases or tickets | A table under the account with a status, named for tickets, cases or support, or with a subject and a body |
| Activities | Calls, emails, meetings and the like, tied to an account or a deal |
| Owners | The team other tables point to (reps, account managers), with their emails |

A business that sells to people, with no table of companies, loads its customers as contacts. Tables no standard object
holds (subscriptions, invoices, usage) are left out, and the README names each one. Every column with no standard home
is carried whole as a custom field or property, created before the load.

The built-in SaaS and sales shape fills every object: customers and prospects, their people, won, lost and open deals,
support tickets and the sales team's calls, emails and meetings.

## Salesforce

### What is in the zip

| File | What it holds |
|---|---|
| `01_User.csv`, `02_Account.csv`, ... | One CSV per object, numbered in load order |
| `custom_fields.json` | Every field to create first, with its type, length, places and picklist values |
| `manifest.json` | The load order, each file's lookups, the values that were worked out and how, each stage's probability and forecast category |
| `README.md` | How to load, and what was left out and why |
| `load_salesforce.py` | The loader |

### Objects and load order

User, Account, Contact, Opportunity, Case, then Task and Event. A record's parents always load before it.

Every record carries the dataset's own key in `Misata_Id__c`, an external id. Every lookup names its parent by that key:
a contact's account is the column `Account.Misata_Id__c`, and an activity on a deal is
`Opportunity:What.Misata_Id__c`. The files upsert with Bulk API 2.0 in order, so each lookup resolves to a record loaded
earlier, and a second load updates the same records instead of adding new ones.

### Values a new org accepts

- **Picklists** use the values a new org ships with: stages, account types, industries, lead sources, case statuses,
  priorities and origins, task and event types. A column fills a standard picklist only when every one of its values
  has a place there. Otherwise it is kept as a custom field of its own, so nothing it says is lost.
- **Opportunity stages.** The dataset's stages are placed on a new org's stages in order: its won stage on Closed Won,
  its lost stage on Closed Lost. Probability and forecast category follow the stage, as a new org sets them.
- **Account type** is Customer - Direct for a company with a contract, and Prospect for one without.
- **Activities.** A meeting with a duration loads as an Event; every other activity as a Task, Completed when it is
  dated on or before the data's last day and Not Started after it. Each sits on its opportunity, else on its account.
- **Required fields the dataset does not state** are worked out and listed in the manifest. An opportunity's name is its
  account's name and the month the deal opened. An open deal's close date is its creation plus the median time the
  dataset's closed deals took.
- Dates are `YYYY-MM-DD` and times UTC.

### Loading

The loader needs Python 3.9 or later and `requests`. It reads the org from the environment, never prints it, and runs
from the unzipped folder:

```bash
pip install requests
SF_INSTANCE_URL=https://<my-domain>.my.salesforce.com SF_ACCESS_TOKEN=... \
  python load_salesforce.py --create-fields
```

For an External Client App with the client credentials flow, set `SF_DOMAIN_URL`, `SF_CLIENT_ID` and
`SF_CLIENT_SECRET` instead.

Before anything loads, the loader reads the org. With `--create-fields` it creates the custom fields through the Tooling
API and gives the loading user access to them in a permission set named Misata_Load; without it, it lists any field
still missing and stops. It checks every picklist value against the org's own: a value the org lacks stops the load,
or is left empty with `--blank-invalid`. Then it upserts each file in order. A job with failed rows stops the load, and
its rows are saved under `failed/` with Salesforce's error on each.

**Owners.** `01_User.csv` lists the people who own records. By default (`--owners match`) each is matched to an active
user with the same email, and a record whose owner has no user is owned by the loading user. `--owners create` creates
the users (a licence each; `--notify-email` names the one address their welcome emails go to, and `--profile` their
profile). `--owners me` gives everything to the loading user.

### What a load cannot set

- **Stage and field history** are written by Salesforce as records change. The time a deal reached each stage is
  carried in its own fields instead.
- **Created dates** are set on accounts, contacts and opportunities only when the org has "Set Audit Fields upon Record
  Creation" turned on for the loading user. The loader checks, and the date is kept in its own field either way.
- **Storage.** Most records count 2 KB against data storage. The README and manifest give the export's total; a
  Developer Edition org holds 5 MB. For more, build with fewer accounts or load into a sandbox or trial org.

To use Data Loader or another tool instead, upsert each file in order on `Misata_Id__c`.

## HubSpot

### What is in the zip

| File | What it holds |
|---|---|
| `01_companies.csv`, `02_contacts.csv`, ... | One CSV per object, numbered in load order |
| `properties.json` | The property group `misata` and every property to create first |
| `owners.csv` | The people who own records, with their emails |
| `imports/` | An import request per file, for HubSpot's imports API |
| `manifest.json` | The load order, each file's associations, the pipelines and the values that were worked out |
| `README.md` | How to load, and what was left out and why |
| `load_hubspot.py` | The loader |

### Objects and load order

Companies, contacts, deals, tickets, then calls, emails, meetings and tasks.

Every record carries the dataset's own key in the property `misata_id`, unique on the objects that upsert on it. Each
`association:<object>` column names the record it belongs to by that key. Associations use HubSpot's own types: a
contact, deal or ticket gets its company as its primary company, and an activity is associated with the company and
the deal it names.

### Values a new portal accepts

- **Deals** go to the default sales pipeline: Appointment Scheduled, Qualified To Buy, Presentation Scheduled, Decision
  Maker Bought-In, Contract Sent, Closed Won, Closed Lost. The dataset's stages are placed on them in order.
- **Tickets** go to the default support pipeline: New, Waiting on contact, Waiting on us, Closed. Priorities are LOW,
  MEDIUM and HIGH.
- **Lifecycle stage.** A company with a contract is a customer, and one without is an opportunity.
- **Calls** are outbound, completed when dated on or before the data's last day and queued after it. Where the dataset
  records them, each has its duration and one of HubSpot's default call outcomes (connected, no answer, left voicemail
  and so on).
- **Emails** are sent, or bounced or failed where the dataset says so. **Meetings** are completed when past and
  scheduled when ahead, or no-show, cancelled or rescheduled where the dataset says so. Any other kind of activity is a
  task.

### Loading

The loader needs Python 3.9 or later and `requests`. Run it from the unzipped folder with a private app's access token
that has the CRM object, schema and owner scopes:

```bash
pip install requests
HUBSPOT_TOKEN=... python load_hubspot.py --create-properties
```

With `--create-properties` it creates the properties the files need; without it, it lists any that are missing and
stops. It then reads the portal's pipelines, owners and property options. Stages are matched by id,
else by label, and the load stops on a stage it cannot place. Dropdown values such as industry and lifecycle stage are
matched to the portal's options by label. Owners are matched by email; a person with no user in the portal leaves the
record unassigned and is listed. Then each file is upserted on `misata_id` in order, and each record is associated
with the records its `association:` columns name.

Activities are created once. The ids HubSpot gave them are kept in `hubspot_ids.json` in the folder, so a second run
adds nothing twice. A batch with errors stops the load, and its rows are saved under `failed/`.

The import requests in `imports/` set properties only. Owners and associations need the loader.

### What a load cannot set

- **Stage history.** The time a deal entered each stage is worked out by HubSpot as a deal moves. Each stage's time is
  carried in its own property instead.
- **Create dates** of contacts and companies are set by HubSpot when they load. The dataset's dates are kept in their
  own properties. A deal's and a ticket's create date is set from the data.

## Load from the API

Misata can run either loader itself, so your product loads a portal or an org with one call and nothing to install.
The same files go in the same order with the same checks as above.

```bash
# plan: each object and how many records, and with a token, the properties it would create first
curl -X POST https://api.misata.studio/v1/datasets/$ID/load -H "Authorization: Bearer $MISATA_KEY" \
  -H "Content-Type: application/json" -d '{"target": "hubspot", "token": "'$HUBSPOT_TOKEN'"}'

# load
curl -X POST https://api.misata.studio/v1/datasets/$ID/load -H "Authorization: Bearer $MISATA_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target": "salesforce", "token": "'$SF_ACCESS_TOKEN'", "instance_url": "https://acme.my.salesforce.com", "apply": true}'
```

| Field | What it is |
|---|---|
| `target` | `hubspot` or `salesforce`. |
| `token` | A HubSpot private app's access token, or a Salesforce access token. Needed to read the CRM and to load. |
| `instance_url` | Salesforce only: the org's `https://<domain>.my.salesforce.com`, as the sign-in returns it. |
| `create_fields` | Create the properties or fields the files need first (default `true`). With `false`, a missing one stops the load and is named. |
| `apply` | `false` (default) plans and writes nothing; `true` loads. |

A load answers with each object and how many records it wrote, and the loader's own lines. Records are upserted on
the Misata id, so loading again updates the same records. HubSpot activities are created once per portal: a second load
adds none again. Salesforce owners are matched to active users by email, and a record whose owner has no user is owned
by the loading user.

The token is used for that one call and is never stored or logged. Misata reaches only `api.hubapi.com` or your org's
own `*.salesforce.com` address, over HTTPS, and follows no redirect. A refused token, a missing scope or a picklist
value the org lacks comes back as a `422` naming the field to change. One call loads up to 100,000 records; past that,
take the export and run its loader where you are.

## Bring an org's own shape

To build data for an org or portal that already has its own fields, import its structure instead of describing it. On
**New dataset**, choose **Import a structure** and add:

- **Salesforce:** an object's describe JSON (`sobjects/<Object>/describe`, or `describeSObjects` for several), custom
  objects included. Each object becomes a table, picklists keep their active values, and lookups become links between
  the objects you include. Fields Salesforce keeps for itself (audit and calculated fields) are left out, and said.
- **HubSpot:** an object's properties (`GET /crm/v3/properties/<object>`, the file named for the object, such as
  `deals.json`) or object schemas (`GET /crm/v3/schemas`), custom objects included. Dropdowns keep their options, and
  each record's primary association becomes a link. Calculated and read-only properties are left out, and counted.

Then state the numbers on those tables as with any [imported structure](/docs/learn/your-own-tables).