Everything You Need to Know About Salesforce Big Objects

Big Objects are Salesforce’s answer to handling massive amounts of data on the Force platform. These Big Objects are unlike Salesforce standard or custom objects, which have data size limits based upon the Salesforce edition, aka, storage limit, of the Salesforce organization. Today’s mission is to dive deeply into Big Objects, including how to use Salesforce Big Objects, Big Object limitations, and best practices when creating Big Objects.
Clear blue sky with white clouds filling half of the sky.

Storing Data On-Platform: Objects vs. Big Objects

Salesforce has three ways to store data on the platform:

  • Standard Objects
  • Custom Objects
  • Big Objects.

Most of the data in a CRM system is stored in Salesforce standard and custom objects. These types of data are commonly used for everyday customer transactions. However, when it comes to storing data with millions or billions of records, Big Objects can be a better alternative. To fully understand the benefits of Big Objects, we first need to define the two different object types, Standard and Custom Objects, and highlight the critical differences between them, especially when it comes to their impact on data storage.

Standard Objects 

Standard Objects are the built-in structures provided by Salesforce. They are designed to handle everyday business operations such as managing user data, sales leads, and customer interactions. Standard objects include predefined entities like Accounts, Contacts, Leads, and Opportunities. These objects are integral to Salesforce and are tailored to cover common CRM functionalities across various industries.

Custom Objects 

Salesforce custom objects expand data model functionality by allowing you to create new database tables, aka objects. These are created each time a managed package from the Salesforce AppExchange is installed into an org or by a Salesforce administrator. Custom Objects allow organizations to store information unique to their processes. For instance, a university could create a custom object to manage student records, courses, or faculty interactions, which are not available as standard objects.  

Standard and Custom Objects Are Meant for Lower Data Volumes

Salesforce’s standard and custom objects are not designed to handle massive data volumes. When data volumes get high, a Salesforce org will see performance issues, particularly in reporting. This is in addition to increased data storage costs! Any object containing over 1 million records can generally cause performance degradation, especially in combination with many custom fields. When data volumes on Salesforce get high, there are two options:

  1. Archive data off of the Salesforce platform
  2. Move data to a Big Object

Big Objects

A Big Object refers to an advanced data structure specifically designed to store and manage extremely large datasets efficiently. These structures are essential in dealing with the complexities of big data, enabling scalable storage, improved query performance, and robust data analysis capabilities across various technology ecosystems. 

However, within the Salesforce environment, a “Big Object” is defined more narrowly. Salesforce Big Objects are customized solutions tailored to archive and analyze historical data on a massive scale within the Salesforce platform. These are not typical database tables; they are structured to provide high-volume storage solutions that maintain performance without affecting the operational database. 

Salesforce custom and standard objects are designed to capture transactional data, such as a customer’s interactions with a business’s sales and support team. This data is updated frequently, allowing a single record to connect to hundreds or thousands of related records. Conversely, Big Objects are created to aggregate and maintain massive data sets, and the data contained in Big Objects is rarely edited. Instead, Big Objects tend to grow as additional records are added to offset data storage costs and improve overall organization performance. 

FieldHistoryArchive is the most common Big Object, and Salesforce automatically creates this object to contain record field history.

Below is a comparative table that outlines the key characteristics and differences between these three types of Salesforce objects:

Big Objects Comparison

Primary Types of Big Objects: Standard & Custom

  1. Standard Big Objects: Standard Big Objects are pre-configured within various platforms to cater to general use cases. These objects are designed to integrate seamlessly into any system, providing robust solutions for common data challenges without the need for additional customization. 
  2. Custom Big Objects: Custom Big Objects offer a tailored approach, allowing organizations to design and structure data solutions that align with their specific business needs. 

Both types of Big Objects have common use cases, including large-scale data archiving, where long-term data retention is critical, comprehensive analytics on historical data to drive business decisions, and regulatory compliance, where vast amounts of data need to be securely maintained and accessed. Standard Big Objects are often employed in industries with common data practices, such as healthcare for patient records. Custom Big Objects are favored in fields like digital marketing, where data needs are continually evolving and highly specific.

Why You Should Use a Big Object

There are two common use cases that drive adoption of Salesforce Big Objects: 

Store Big Data for Historical Analytics

Big Objects store billions of records, all in a way where the data is accessible natively inside Salesforce. This lets users create reports on this data without requiring additional off-platform storage or integrations. As a result, Big Objects can help a business analyze customer trends over a long period. 

Key Features of Salesforce Big Object Storage:

  • Scalability: Salesforce Big Objects can scale to store billions of records. This scalability is vital for organizations that accumulate large amounts of data over time and need a sustainable way to manage it.
  • Performance: By using Big Objects, businesses can ensure that the performance of their everyday Salesforce applications remains unaffected, even as they handle large datasets. Big Objects are optimized for high-volume data storage and query operations on large datasets.
  • Archiving: One of the primary uses of Big Objects is for data archiving. Organizations can archive old transactional data that must be retained for regulatory compliance or historical analysis. This offloads the data from the main CRM environment, helping to maintain system performance and reduce storage costs.
  • Data Accessibility: Despite being designed for large-scale data, Big Objects are integrated within Salesforce, making the data accessible through Salesforce’s native tools. This allows for seamless querying and reporting, even on large historical datasets.

Salesforce Big Object storage provides a specialized, highly efficient means of storing and managing large volumes of data within the Salesforce platform. This capability is essential for companies that need to maintain large historical datasets for compliance, analytical, or operational needs, enabling them to leverage their data effectively without sacrificing performance.

Reduction of Data Costs

The per-record storage rate for a Big Object record is significantly lower than the per-record cost for a Salesforce organization exceeding its data storage limit. When this article was published, the general Salesforce pricing guide listed Big Objects as a $16,800 yearly investment or .0003 US cents per record. Alternatively, while not publicly published, Salesforce’s data storage rate is rumored to be $125 per month for 500 MB of extra data storage. Let’s do some simple math:

Big Object Cost:

.0003 cents per record 

Salesforce Standard or Custom Object Cost: 

.006 cents per record 

Annual cost per data storage block:

$1,500 to contain 250,000 records 

= .006 cents per record 

With such a large cost discrepancy, it should be no surprise that Salesforce Big Objects are rapidly being adopted to support both data storage and data cost use cases!

Big Object Technical Consideration: Index

Unlike Salesforce’s custom objects, you determine what fields should be used as an index on Big Object creation. This is perhaps the most critical technical consideration as it determines the future use of the Big Object. In addition, the index determines how the data stored in the object is filtered and queried. 

An index is set upon Big Object creation, and it can not change later

Consider the implications of this statement. First, a Big Object can contain billions of records. An index that does not promote efficient data filtering or searching could negate the value of using this technology.

Example: A bad index

This index uses an Account lookup as the 1st custom index. Here’s why this could cause a big problem in the future:

There are three records in my Big Object today:

  • Initech  
  • Massive Dynamic
  • Umbrella Corporation

When a new record is created – for this example, Stark Industries – the record will be inserted into the index between Massive Dynamic and Umbrella Corporation. As the first index field is also a lookup, we could also have records that share the same first index field! This means that we have no efficient way to search the index to see what records are new or what records have changed! 

Example: A good index 

This index is designed for queries with our first indexed field based on the record creation data, creating an iterator. For example, we can then query records created since a specific date. The second part of our index is a unique case number, which functions as a unique identifier even if we end up with two records created on the same date and time. Finally, the Account lookup and phone number are captured much later in our index. 

A good Big Object index must contain two core components:

  1. An iterator, like a timestamp
  2. A uniqueness indicator, like an ID 

CapStorm Supports Full Backups of Salesforce Big Objects

When Big Objects are used, the data stored within them may need to be backed up to ensure the organization has a comprehensive Salesforce business continuity plan. At the time this article was published, Salesforce’s Backup Solution did not support the backup of Big Objects. FieldHistoryArchive, the most common Big Object, was not supported by Ownbackup (Knowledge Base: Salesforce Objects Included and Excluded in Backups), Grax, Spanning, and others.

CapStorm supports the replication of FieldHistoryArchive and other Big Objects. The CopyStorm application supports incremental replication from Salesforce Big Objects for those that have well-constructed indexes. This means that added or modified records are replicated during each backup, and the Big Object backups can run at a high frequency. For Big Objects that lack a uniqueness indicator and an iterator, CapStorm supports a full extract of these records.

Want to learn more? Check out a short demo video below:

For even more information on CapStorm’s Salesforce backup solutions, talk to one of our experts today!



Rebecca Gray

Rebecca Gray

Rebecca is 5 year Salesforce fanatic and certified Salesforce Admin, Service Cloud Consultant, Sales Cloud Consultant, and App Builder. She volunteers in the Salesforce community, leading the Saint Louis, MO Salesforce Admin Group and is a former Lightning Champion. In her day job, Rebecca supports Customer Success, helping CapStorm customers achieve their goals for Salesforce data management.

About CapStorm

CapStorm is the most technologically advanced Salesforce data management platform on the market. Billions of records per day flow through CapStorm software, and our solutions are used in every industry from credit cards, telecom providers, insurance agencies, global banks and energy providers.

Recent Posts

Follow Us

Become a CapStorm Insider

Become a CapStorm Insider

Subscribe to the CapStorm Forecast

Name
This field is for validation purposes and should be left unchanged.