Secondary Indexing and Bloom Filters in Spark.

Bloom filters are probabilistic data structures used to skip reading data that definitely doesn't match a filter.

Spark can use Bloom Filter Joins to significantly prune the data being shuffled. It creates a filter from the small side of a join and applies it to the large side before the shuffle.