Firestore for Rust

Library provides a simple API for Google Firestore based on the official gRPC API:

• Fluent high-level and strongly typed API, the only public API of this library;
• Create or update documents using Rust structures and Serde;
• Support for:
  • Querying/streaming docs/objects;
  • Listing documents/objects (stream_all follows the page tokens for you);
  • Listening changes from Firestore;
  • Transactions;
  • Aggregated queries: count, sum, avg;
  • Streaming batch writes with automatic throttling to stay under the Firestore write rate limits;
  • K-nearest neighbor (KNN) vector search with Euclidean, Cosine and Dot Product measures;
  • Query cursors and partition queries;
  • Collection group queries and listing collection IDs;
  • Server side document transforms: increment, array append/remove, server timestamp;
  • Update/delete preconditions;
  • Consistency selectors to read at a given time or inside a transaction;
  • Explaining queries;
  • Request tags to attribute Firestore usage;
• Full async based on Tokio runtime;
• Macros that help you use your structure fields as Firestore field paths;
• Implements own Serde serializer to Firestore protobuf values;
• Support for multiple database IDs;
• Support for extended datatypes:
  • Firestore timestamp as a FirestoreTimestamp type or with #[serde(with)] attributes (based on jiff);
  • Lat/Lng;
  • References;
  • Explicit nulls;
• Caching support for collections and documents:
  • In-memory cache;
  • Persistent cache;
• Works with the Firestore emulator through FIRESTORE_EMULATOR_HOST, no credentials needed;
• Google client based on gcloud-sdk library that automatically detects GCE environment or application default accounts for local development;

Quick start

Cargo.toml:

[dependencies]
firestore = "0.55"

Crypto provider error

Depends on your other dependencies you may see the error like:

no process-level CryptoProvider available -- call CryptoProvider::install_default() before this point

This is because the TLS providers are not installed by default and you can choose different. The easiest way to fix is just to include one of the provider, for example:

[dependencies]
rustls = "0.23"

If you have multiple you may need to call CryptoProvider::install_default() before using the Firestore client, e.g.:

rustls::crypto::ring::default_provider().install_default().expect("Failed to install rustls crypto provider");

Running the examples

All examples available in the examples directory.

To run an example with environment variables:

PROJECT_ID=<your-google-project-id> cargo run --example crud

Firestore database client instance and lifecycle

To create a new instance of Firestore client you need to provide at least a GCP project ID. It is not recommended creating a new client for each request, so it is recommended to create a client once and reuse it whenever possible. Cloning instances is much cheaper than creating a new one.

The client is created using the Firestore::new method:

fn config_env_var(name: &str) -> Result<String, String> {
    std::env::var(name).map_err(|e| format!("{name}: {e}"))
}
async fn example() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
use firestore::*;

// Create an instance
let db = FirestoreDb::new(&config_env_var("PROJECT_ID")?).await?;
Ok(())
}

This is the recommended way to create a new instance of the client, since it automatically detects the environment and uses credentials, service accounts, Workload Identity on GCP, etc. Look at the section below Google authentication for more details.

In cases if you need to create a new instance explicitly specifying a key file, you can use:

use firestore::*;
fn config_env_var(name: &str) -> Result<String, String> {
    std::env::var(name).map_err(|e| format!("{name}: {e}"))
}
async fn example() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
FirestoreDb::with_options_service_account_key_file(
    FirestoreDbOptions::new(config_env_var("PROJECT_ID")?.to_string()),
    "/tmp/key.json".into(),
)
.await?
;
Ok(())
}

or if you need even more flexibility you can use a preconfigured token source and scopes with:

use firestore::*;
fn config_env_var(name: &str) -> Result<String, String> {
    std::env::var(name).map_err(|e| format!("{name}: {e}"))
}
async fn example() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
FirestoreDb::with_options_token_source(
    FirestoreDbOptions::new(config_env_var("PROJECT_ID")?.to_string()),
    gcloud_sdk::GCP_DEFAULT_SCOPES.clone(),
    gcloud_sdk::TokenSourceType::File("/tmp/key.json".into()),
)
.await?
;
Ok(())
}

Firebase supports multiple databases per project now, so you can specify the database ID in the options:

use firestore::*;
async fn example() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
FirestoreDb::with_options(
    FirestoreDbOptions::new("your-project-id".to_string())
        .with_database_id("your-database-id".to_string()),
)
.await?
;
Ok(())
}

Fluent API

The Fluent API is the only public API of this library. Everything starts from db.fluent():

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    some_id: String,
    some_string: String,
    one_more_string: String,
    some_num: u64,
}
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
use firestore::*;

const TEST_COLLECTION_NAME: FirestoreCollectionId = FirestoreCollectionId::from_static("test");

let my_struct = MyTestStructure {
    some_id: "test-1".to_string(),
    some_string: "Test".to_string(),
    one_more_string: "Test2".to_string(),
    some_num: 42,
};

// Create
let object_returned: MyTestStructure = db
    .fluent()
    .insert()
    .into(TEST_COLLECTION_NAME)
    .document_id(&my_struct.some_id)
    .object(&my_struct)
    .execute()
    .await?;

// Update or Create
// (Firestore supports creating documents with update if you provide the document ID).
let object_updated: MyTestStructure = db
    .fluent()
    .update()
    .fields(paths!(MyTestStructure::{some_num, one_more_string}))
    .in_col(TEST_COLLECTION_NAME)
    .document_id(&my_struct.some_id)
    .object(&MyTestStructure {
        some_num: my_struct.some_num + 1,
        one_more_string: "updated-value".to_string(),
        ..my_struct.clone()
    })
    .execute()
    .await?;

// Get object by id
let find_it_again: Option<MyTestStructure> = db
    .fluent()
    .select()
    .by_id_in(TEST_COLLECTION_NAME)
    .obj()
    .one(&my_struct.some_id)
    .await?;

// Delete data
db.fluent()
    .delete()
    .from(TEST_COLLECTION_NAME)
    .document_id(&my_struct.some_id)
    .execute()
    .await?;
Ok(())
}

The low level “support” traits were made crate private in v0.52.0. If you used them, see the migration guide for the fluent replacement of every removed method.

Alongside it, the library provides core functionality that is public API in its own right:

• Batch writes: db.create_simple_batch_writer(), db.create_streaming_batch_writer()
• Transactions: db.begin_transaction(), db.run_transaction(), and the FirestoreTransactionOps trait implemented by both FirestoreTransaction and FirestoreTransactionData
• Listeners: db.create_listener(), and the FirestoreResumeStateStorage trait for custom resume token storage
• Caching: the FirestoreCacheBackend / FirestoreCacheDocsByPathSupport traits for custom cache backends
• Dynamic documents: FirestoreDb::serialize_map_to_doc(), FirestoreDb::serialize_to_doc() and FirestoreDb::deserialize_doc_to(), used together with the fluent .document(...) builders

Querying

The library supports rich querying API with filters, ordering, pagination, etc.

use firestore::*;
use futures::stream::BoxStream;
use futures::TryStreamExt;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    some_id: String,
    some_string: String,
    one_more_string: String,
    some_num: u64,
    created_at: FirestoreTimestamp,
}
const TEST_COLLECTION_NAME: &str = "test-query";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// Query as a stream our data
let object_stream: BoxStream<FirestoreResult<MyTestStructure>> = db
    .fluent()
    .select()
    .fields(
        paths!(MyTestStructure::{some_id, some_num, some_string, one_more_string, created_at}),
    ) // Optionally select the fields needed
    .from(TEST_COLLECTION_NAME)
    .filter(|q| {
        // Fluent filter API example
        q.for_all([
            q.field(path!(MyTestStructure::some_num)).is_not_null(),
            q.field(path!(MyTestStructure::some_string)).eq("Test"),
            // Sometimes you have optional filters
            Some("Test2")
                .and_then(|value| q.field(path!(MyTestStructure::one_more_string)).eq(value)),
        ])
    })
    .order(|o| o.fields([o.field(path!(MyTestStructure::some_num)).desc()]))
    .obj() // Reading documents as structures using Serde gRPC deserializer
    .stream_query_with_errors()
    .await?;

let as_vec: Vec<MyTestStructure> = object_stream.try_collect().await?;
println!("{:?}", as_vec);
Ok(())
}

Use:

• q.for_all for AND conditions
• q.for_any for OR conditions (Firestore has just recently added support for OR conditions)

You can nest q.for_all/q.for_any.

Ordering

.order() takes a closure that receives an order builder and returns a Vec, the same shape as .filter() and .transforms(). List multiple fields to sort by more than one, in priority order:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    some_id: String,
    some_num: u64,
}
const TEST_COLLECTION_NAME: &str = "test-query";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let ordered: Vec<MyTestStructure> = db
    .fluent()
    .select()
    .from(TEST_COLLECTION_NAME)
    .order(|o| {
        o.fields([
            o.field(path!(MyTestStructure::some_num)).desc(),
            o.field(path!(MyTestStructure::some_id)).asc(),
        ])
    })
    .obj()
    .query()
    .await?;
let _ = ordered;
Ok(())
}

An entry can be conditional, since field(..).asc()/.desc() return Option<FirestoreQueryOrder> and None is dropped:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    some_id: String,
    some_num: u64,
}
const TEST_COLLECTION_NAME: &str = "test-query";
async fn example(db: FirestoreDb, sort_by_num: bool) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let ordered: Vec<MyTestStructure> = db
    .fluent()
    .select()
    .from(TEST_COLLECTION_NAME)
    .order(|o| {
        o.fields([sort_by_num.then(|| o.field(path!(MyTestStructure::some_num)).desc())])
    })
    .obj()
    .query()
    .await?;
let _ = ordered;
Ok(())
}

Each call to .order() replaces any ordering set by a previous call; it does not add to it. An empty result (every entry conditional and every condition false) clears the ordering entirely.

If the query also uses start_at/end_at, the cursor’s value count must match the number of ordered fields - this is not enforced by the library, so a mismatch is a runtime error from Firestore rather than a compile-time one.

.order_by([(path!(..), FirestoreQueryDirection::Descending)]) still works but is deprecated in favor of .order().

Get and batch get support

use firestore::*;
use futures::stream::BoxStream;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    some_id: String,
}
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb, my_struct: MyTestStructure) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let find_it_again: Option<MyTestStructure> = db
    .fluent()
    .select()
    .by_id_in(TEST_COLLECTION_NAME)
    .obj()
    .one(&my_struct.some_id)
    .await?;

let object_stream: BoxStream<(String, Option<MyTestStructure>)> = db
    .fluent()
    .select()
    .by_id_in(TEST_COLLECTION_NAME)
    .obj()
    .batch(vec!["test-0", "test-5"])
    .await?;
Ok(())
}

Nested collections

You can work with nested collections specifying path/location to a parent for documents. parent_path and the builder’s at also validate the collection name they are given, so a FirestoreCollectionId or a bad raw &str is caught in the same call rather than at the server.

use firestore::*;
use futures::stream::BoxStream;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyParentStructure { some_id: String }
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyChildStructure { some_id: String }
const TEST_PARENT_COLLECTION_NAME: &str = "nested-test";
const TEST_CHILD_COLLECTION_NAME: &str = "test-childs";
async fn example(
    db: FirestoreDb,
    parent_struct: MyParentStructure,
    child_struct: MyChildStructure,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// Creating a parent doc
db.fluent()
    .insert()
    .into(TEST_PARENT_COLLECTION_NAME)
    .document_id(&parent_struct.some_id)
    .object(&parent_struct)
    .execute::<()>()
    .await?;

// The doc path where we store our children
let parent_path = db.parent_path(TEST_PARENT_COLLECTION_NAME, parent_struct.some_id)?;

// Create a child doc
db.fluent()
    .insert()
    .into(TEST_CHILD_COLLECTION_NAME)
    .document_id(&child_struct.some_id)
    .parent(&parent_path)
    .object(&child_struct)
    .execute::<()>()
    .await?;

// Listing children
println!("Listing all children");

let objs_stream: BoxStream<MyChildStructure> = db
    .fluent()
    .list()
    .from(TEST_CHILD_COLLECTION_NAME)
    .parent(&parent_path)
    .obj()
    .stream_all()
    .await?;
let _ = objs_stream;
Ok(())
}

Complete example available here.

You can nest multiple levels of collections using at():

use firestore::*;
const TEST_PARENT_COLLECTION_NAME: &str = "nested-test";
const TEST_CHILD_COLLECTION_NAME: &str = "test-childs";
const TEST_GRANDCHILD_COLLECTION_NAME: &str = "test-grandchilds";
fn example(db: &FirestoreDb) -> Result<(), Box<dyn std::error::Error>> {
let parent_path = db
    .parent_path(TEST_PARENT_COLLECTION_NAME, "parent-id")?
    .at(TEST_CHILD_COLLECTION_NAME, "child-id")?
    .at(TEST_GRANDCHILD_COLLECTION_NAME, "grand-child-id")?;
let _ = parent_path;
Ok(())
}

Transactions

To manage transactions manually you can use db.begin_transaction(), and then the Fluent API to add the operations needed in the transaction.

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { some_id: String, some_string: String }
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let mut transaction = db.begin_transaction().await?;

db.fluent()
    .update()
    .fields(paths!(MyTestStructure::{
      some_string
    }))
    .in_col(TEST_COLLECTION_NAME)
    .document_id("test-0")
    .object(&MyTestStructure {
        some_id: format!("test-0"),
        some_string: "UpdatedTest".to_string(),
    })
    .add_to_transaction(&mut transaction)?;

db.fluent()
    .delete()
    .from(TEST_COLLECTION_NAME)
    .document_id("test-5")
    .add_to_transaction(&mut transaction)?;

transaction.commit().await?;
Ok(())
}

You may also execute transactions that automatically retry with exponential backoff using run_transaction.

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { test_string: String }
const TEST_COLLECTION_NAME: &str = "test";
const TEST_DOCUMENT_ID: &str = "test_doc_id";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
db.run_transaction(|db, transaction| {
    Box::pin(async move {
        let mut test_structure: MyTestStructure = db
            .fluent()
            .select()
            .by_id_in(TEST_COLLECTION_NAME)
            .obj()
            .one(TEST_DOCUMENT_ID)
            .await?
            .expect("Missing document");

        // Perform some kind of operation that depends on the state of the document
        test_structure.test_string += "a";

        db.fluent()
            .update()
            .fields(paths!(MyTestStructure::{
             test_string
            }))
            .in_col(TEST_COLLECTION_NAME)
            .document_id(TEST_DOCUMENT_ID)
            .object(&test_structure)
            .add_to_transaction(transaction)?;

        Ok(())
    })
})
.await?;
Ok(())
}

See the complete example available here.

Please note that Firestore doesn’t support creating documents in the transactions (generating document IDs automatically), so you need to use update() to implicitly create documents and specifying your own IDs.

Document transformations

The library supports server side document transformations in transactions and batch writes:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { some_num: i32, some_array: Vec<i32> }
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb, my_obj: MyTestStructure) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let mut transaction = db.begin_transaction().await?;
// Only transformation
db.fluent()
    .update()
    .in_col(TEST_COLLECTION_NAME)
    .document_id("test-4")
    .transforms(|t| {
        // Transformations
        t.fields([
            t.field(path!(MyTestStructure::some_num)).increment(10),
            t.field(path!(MyTestStructure::some_array))
                .append_missing_elements([4, 5]),
            t.field(path!(MyTestStructure::some_array))
                .remove_all_from_array([3]),
        ])
    })
    .only_transform()
    .add_to_transaction(&mut transaction)?; // or add_to_batch

// Update and transform (in this order and atomically):
db.fluent()
    .update()
    .in_col(TEST_COLLECTION_NAME)
    .document_id("test-5")
    .object(&my_obj) // Updating the objects with the fields here
    .transforms(|t| {
        // Transformations after the update
        t.fields([t.field(path!(MyTestStructure::some_num)).increment(10)])
    })
    .add_to_transaction(&mut transaction)?; // or add_to_batch
Ok(())
}

Select aggregate functions

The library supports the aggregation functions for the queries:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyAggTestStructure { counter: usize }
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let objs: Vec<MyAggTestStructure> = db
    .fluent()
    .select()
    .from(TEST_COLLECTION_NAME)
    .aggregate(|a| a.fields([a.field(path!(MyAggTestStructure::counter)).count()]))
    .obj()
    .query()
    .await?;
let _ = objs;
Ok(())
}

Update/delete preconditions

The library supports the preconditions:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    some_num: u64,
}
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let object_updated: MyTestStructure = db
    .fluent()
    .update()
    .fields(paths!(MyTestStructure::{some_num}))
    .in_col(TEST_COLLECTION_NAME)
    .precondition(FirestoreWritePrecondition::Exists(true))
    .document_id("test-1")
    .object(&MyTestStructure { some_num: 1 })
    .execute()
    .await?;
let _ = object_updated;
Ok(())
}

Explaining the query

The library supports the query explanation:

use firestore::*;
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
db.fluent()
    .select()
    .from(TEST_COLLECTION_NAME)
    .explain()
    // or use explain_with_options if you want to provide additional options like analyze which run query to gather additional statistics
    // .explain_with_options(FirestoreExplainOptions::new().with_analyze(true))
    .stream_query_with_metadata()
    .await?;
Ok(())
}

Listening the document changes on Firestore

To help to work with asynchronous event listener the library supports high level API for listening the events from Firestore on a separate thread:

The listener implementation needs to be provided with a storage for the last received token for specified targets to be able to resume listening the changes from the last handled token and to avoid receiving all previous changes.

The library provides basic implementations for storing the tokens but you can implement your own more sophisticated storage if needed:

• FirestoreTempFilesListenStateStorage - resume tokens stored as temporary files on local FS;
• FirestoreMemListenStateStorage - in memory storage backed by HashMap (with this implementation if you restart your app, you will receive all notifications again);

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { some_id: String }
const TEST_COLLECTION_NAME: &str = "test-listen";
const TEST_TARGET_ID_BY_QUERY: FirestoreListenerTarget = FirestoreListenerTarget::new(42_u32);
const TEST_TARGET_ID_BY_DOC_IDS: FirestoreListenerTarget = FirestoreListenerTarget::new(17_u32);
async fn example(db: FirestoreDb, doc_id1: String, doc_id2: String) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let mut listener = db
    .create_listener(
        FirestoreTempFilesListenStateStorage::new(), // or FirestoreMemListenStateStorage or your own implementation
    )
    .await?;

// Adding query listener
db.fluent()
    .select()
    .from(TEST_COLLECTION_NAME)
    .listen()
    .add_target(TEST_TARGET_ID_BY_QUERY, &mut listener)?;

// Adding docs listener by IDs
db.fluent()
    .select()
    .by_id_in(TEST_COLLECTION_NAME)
    .batch_listen([doc_id1, doc_id2])
    .add_target(TEST_TARGET_ID_BY_DOC_IDS, &mut listener)?;

listener
    .start(|event| async move {
        match event {
            FirestoreListenEvent::DocumentChange(ref doc_change) => {
                println!("Doc changed: {:?}", doc_change);

                if let Some(doc) = &doc_change.document {
                    let obj: MyTestStructure =
                        FirestoreDb::deserialize_doc_to::<MyTestStructure>(doc)
                            .expect("Deserialized object");
                    println!("As object: {:?}", obj);
                }
            }
            _ => {
                println!("Received a listen response event to handle: {:?}", event);
            }
        }

        Ok(())
    })
    .await?;

// Wait some events like Ctrl-C, signals, etc
// <put-your-implementation-for-wait-here>

// and then shutdown
listener.shutdown().await?;
Ok(())
}

See complete example in examples directory.

Working on dynamic/document level

Sometimes having static structure may restrict you from working with dynamic data, so there is a way to use Fluent API to work with documents without introducing structures at all.

use firestore::*;
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let object_returned = db
    .fluent()
    .insert()
    .into(TEST_COLLECTION_NAME)
    .document_id("test-1")
    .document(FirestoreDb::serialize_map_to_doc(
        "",
        [
            ("some_id", "test-id".into()),
            ("some_string", "test-value".into()),
            ("some_num", 42.into()),
            (
                "embedded_obj",
                FirestoreValue::from_map([
                    ("inner_some_id", "inner-id-value".into()),
                    ("inner_some_string", "inner-some-value".into()),
                ]),
            ),
            ("created_at", FirestoreTimestamp::now().into()),
        ],
    )?)
    .execute()
    .await?;
let _ = object_returned;
Ok(())
}

Full example available here.

Timestamps support

By default, date/time values serialize as a string to Firestore (while deserialization works from Timestamps and Strings). To store them as native Firestore timestamps there are three options.

• Using std::time::SystemTime directly, with no attribute and no wrapping type, since it is recognised automatically:

use serde::{Deserialize, Serialize};
use std::time::SystemTime;
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    created_at: SystemTime,
    updated_at: Option<SystemTime>,
}

It works in the queries as well:

use firestore::*;
use std::time::SystemTime;
#[derive(Debug, Clone, serde::Deserialize, serde::Serialize)]
struct MyTestStructure { created_at: SystemTime }
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
db.fluent().select().from(TEST_COLLECTION_NAME).filter(|q| { q.for_all([
q.field(path!(MyTestStructure::created_at))
    .less_than_or_equal(SystemTime::now())
]) }).obj::<MyTestStructure>().query().await?;
Ok(())
}

Note that SystemTime cannot carry the instants before the Unix epoch, since serde itself refuses them.

• Using the type FirestoreTimestamp, which needs no attributes:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    created_at: FirestoreTimestamp,
    updated_at: Option<FirestoreTimestamp>,
}

It can be created with FirestoreTimestamp::now(), parsed from a string, and converted from/to std::time::SystemTime:

use firestore::*;
use std::time::SystemTime;
fn example() -> Result<(), Box<dyn std::error::Error>> {
let now = FirestoreTimestamp::now();
let from_system_time: FirestoreTimestamp = SystemTime::now().try_into()?;
let back_to_system_time: SystemTime = now.into();
let _ = (from_system_time, back_to_system_time);
Ok(())
}

Use it in your queries as well, for example:

use firestore::*;
#[derive(Debug, Clone, serde::Deserialize, serde::Serialize)]
struct MyTestStructure { created_at: FirestoreTimestamp }
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
db.fluent().select().from(TEST_COLLECTION_NAME).filter(|q| { q.for_all([
q.field(path!(MyTestStructure::created_at))
    .less_than_or_equal(FirestoreTimestamp::now())
]) }).obj::<MyTestStructure>().query().await?;
Ok(())
}

• Or, if you prefer to keep a plain instant in your model, use FirestoreInstant (an alias for jiff::Timestamp) with #[serde(with)] attributes:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    #[serde(with = "firestore::serialize_as_timestamp")]
    created_at: FirestoreInstant,

    #[serde(default)]
    #[serde(with = "firestore::serialize_as_optional_timestamp")]
    updated_at: Option<FirestoreInstant>,
}

Firestore stores the timestamps with microsecond precision and discards the nanoseconds on write. FirestoreTimestamp truncates to that precision in all of its constructors and conversions, so its values survive a round trip unchanged. A FirestoreInstant or a SystemTime carrying nanoseconds does not, since those are the standard types this library cannot change.

All of them change the representation only for Firestore serialization. FirestoreTimestamp and FirestoreInstant still serialize as a string to JSON, so the same model can be reused for JSON and Firestore, while a plain SystemTime keeps the default serde representation.

Reading Firestore document metadata as struct fields

Firestore provides additional generated fields for each of document you create:

• _firestore_id: Generated document ID (when it is not specified from the client);
• _firestore_created: The time at which the document was created;
• _firestore_updated: The time at which the document was last changed;

To be able to read them the library makes them available as system fields for the Serde deserializer with reserved names, so you can specify them in your structures as:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
    #[serde(alias = "_firestore_id")]
    id: Option<String>,
    #[serde(alias = "_firestore_created")]
    created_at: Option<FirestoreTimestamp>,
    #[serde(alias = "_firestore_updated")]
    updated_at: Option<FirestoreTimestamp>,
    some_string: String,
    one_more_string: String,
    some_num: u64,
}

Complete example available here.

Explicit null value serialization

By default, all Option<> serialized as absent fields, which is convenient for many cases. However sometimes you need to have explicit nulls.

To help with that there are additional attributes implemented for serde(with):

• For any type:

use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
#[serde(default)]
#[serde(with = "firestore::serialize_as_null")]
test_null: Option<String>,
}

• For Firestore timestamps attribute:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure {
#[serde(default)]
#[serde(with = "firestore::serialize_as_null_timestamp")]
test_null: Option<FirestoreInstant>,
}

Document and collection IDs

FirestoreDocumentId and FirestoreCollectionId are validated newtypes for IDs that arrive from outside your process - a request path segment, a JSON body, a runtime-chosen collection name. Construct one where the ID enters your system; a /, an empty string, or ./.. is rejected there instead of silently reaching Firestore or, in the case of a collection name, retargeting the operation at a different collection.

use firestore::*;
fn example() -> Result<(), Box<dyn std::error::Error>> {
let id = FirestoreDocumentId::new("user-42")?;
let collection = FirestoreCollectionId::new("users")?;
let _ = (id, collection);
Ok(())
}

A name known up front - the sort of thing you would today write as const NAME: &str = "..." - can be declared as a validated const or static with from_static. An invalid literal there is a compile error, not a runtime one:

use firestore::*;
const USERS: FirestoreCollectionId = FirestoreCollectionId::from_static("users");

Use from_static for a literal you control, checked at compile time with no Result to handle; use new for a value arriving at runtime, which returns a FirestoreResult.

Both implement AsRef<str>, so a reference drops straight into any call that already takes a document or collection ID or name, with no conversion:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { some_id: String }
async fn example(
    db: FirestoreDb,
    collection: FirestoreCollectionId,
    id: FirestoreDocumentId,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
db.fluent()
    .select()
    .by_id_in(&collection)
    .obj::<MyTestStructure>()
    .one(&id)
    .await?;
Ok(())
}

Store the validated type in your own structs instead of a bare String to carry the proof along with the value; it deserializes through the same validation, so a CreateSession built from an untrusted request body rejects a bad ID before it reaches Firestore:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct CreateSession {
    user_id: FirestoreDocumentId,
}

See examples/crud.rs and examples/nested_collections.rs for a document ID and a parent_path/at pair built from validated IDs, and examples/dynamic_doc_level_crud.rs / examples/caching_dynamic_collections.rs for a collection name chosen at runtime.

Caching

The library supports caching for collections and documents. A Firestore listener keeps the cache up to date when documents change, so updates are propagated across distributed instances automatically.

This avoids reading, and paying for, the same documents repeatedly. It is particularly useful for dictionaries, configuration and other data that changes rarely, and can reduce both cost and latency noticeably.

Caching is opt-in through cargo features:

• caching-memory for an in-memory cache, implemented with the moka cache library;
• caching-persistent for a persistent, disk backed cache, implemented with redb and protobuf.

Usage

Usage

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { some_id: String }
fn config_env_var(name: &str) -> Result<String, String> {
    std::env::var(name).map_err(|e| format!("{name}: {e}"))
}
async fn example() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// Create an instance
let db = FirestoreDb::new(&config_env_var("PROJECT_ID")?).await?;

// Build the cache. This creates an internal Firestore listener, preloads the configured
// collections and starts listening for changes.
let cache = FirestoreCache::memory(&db)
    .preloaded_collection("test-caching")
    .build()
    .await?;

// Read through the cache: served from the cache when possible, from Firestore otherwise.
let my_struct: Option<MyTestStructure> = db
    .read_through_cache(&cache)
    .fluent()
    .select()
    .by_id_in("test-caching")
    .obj()
    .one("test-1")
    .await?;

// Read only from the cache, never contacting Firestore.
let my_struct: Option<MyTestStructure> = db
    .read_cached_only(&cache)
    .fluent()
    .select()
    .by_id_in("test-caching")
    .obj()
    .one("test-1")
    .await?;

cache.shutdown().await?;
let _ = my_struct;
Ok(())
}

For a persistent cache, use FirestoreCache::persistent(&db) and give it a directory with .data_dir("/var/cache/my-app"), which keeps the cache database and the listener resume tokens together.

Listener target IDs are assigned automatically starting at 1000. If your application runs its own listeners, move the cache’s range with .listener_target_base(...) or pin individual collections with .collection_with(name, |c| c.listener_target(...)).

Because load and shutdown take &self, a built cache can be shared directly as Arc<FirestoreMemoryCache> in your application state. FirestoreMemoryCache and FirestorePersistentCache are aliases that save you from spelling out the generic parameters.

shutdown stops the listener and releases the backend’s resources: the in-memory cache drops its documents, and the persistent cache closes its database file, so another cache can be opened over the same directory.

Caching named documents

Caching named documents instead of a whole collection

.collection(name) subscribes the listener to the entire collection, even though it does not preload it - “lazy” only means the initial download is skipped. When you know which documents you care about, say so:

use firestore::*;
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let cache = FirestoreCache::memory(&db)
    .collection_with("configs", |c| {
        c.documents(["site", "billing"]).preload_all()
    })
    .build()
    .await?;
let _ = cache;
Ok(())
}

The listener then watches exactly those documents, so unrelated changes in the collection are never streamed to your process or written into the cache, and preloading reads just those IDs. This is the shape to reach for with configuration, feature flags and reference data.

Such a collection is never listable, whatever its load mode: it holds a chosen subset, so list and query would return a partial answer that looks complete.

Changing cached collections at runtime

Changing the cached collections at runtime

The set of cached collections does not have to be fixed when the cache is built:

use firestore::*;
async fn example(cache: FirestoreMemoryCache) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
cache
    .add_collection(FirestoreCacheCollection::new("currencies").preload_all())
    .await?;

cache.remove_collection("currencies").await?;
Ok(())
}

add_collection downloads the collection first if it is preloaded, publishes it only once it is complete, and then extends the listener - so a listing never observes it half filled, and nothing written during the download is missed. remove_collection does the reverse, and also forgets the collection’s resume token so its listener target ID cannot be reused against a different query. Use remove_collection_at for a sub-collection, whose absolute path a bare name cannot address.

FirestoreDb handles created earlier with read_through_cache or read_cached_only pick both up immediately: they share the cache’s backend rather than a copy of it. A cache can also be built with no collections at all and populated entirely at runtime.

Choosing a cache mode

Choosing a cache mode

• db.read_through_cache(&cache) serves what it can from the cache and goes to Firestore for the rest. This is the mode to reach for by default.
• db.read_cached_only(&cache) never contacts Firestore. Reads by ID return None on a miss, and requests the cache cannot answer completely return an error.

Which operations use the cache:

Load modes and preloading

Load modes, and why listings need preloading

• PreloadNone (.collection(name)): don’t preload anything, just fill the cache while working;
• PreloadAllDocs (.collection_with(name, |c| c.preload_all())): preload all documents in the collection;
• PreloadAllIfEmpty (.collection_with(name, |c| c.preload_all_if_empty())): preload all documents only if the cache is empty. This is useful for the persistent cache; for the memory cache it is the same as PreloadAllDocs, since an in-memory cache always starts empty.

.preloaded_collection(name) picks the appropriate preloading mode for the backend.

A lazily filled collection holds only the documents that happened to be read through it. Answering a list or query from it would return a subset that looks like a complete answer, so the library refuses to do so: read_through_cache quietly falls back to Firestore, and read_cached_only returns an error naming the collection. If partial results are genuinely acceptable, opt in with .incomplete_collection_policy(FirestoreCacheIncompleteCollectionPolicy::PartialResults).

How the cache is updated

How the cache is updated

• When you read a document by ID through the cache and it is not there, it is fetched from Firestore and cached;
• The Firestore listener updates the cache when a document changes, whether the change came from your application or from elsewhere;
• Preloading at startup.

Cached results are eventually consistent: they reflect the last state the listener delivered, so a write may take a moment to show up. Do not cache data that must be read at strong consistency.

Firestore also reports how many documents a target matches. When that disagrees with what a preloaded collection holds - which means changes were missed, typically deletes that happened while the listener was disconnected - the cache drops the collection and has Firestore replay it. The count is only compared when it can be trusted: a collection the cache expires entries from, or fills lazily, is never checked this way.

When Firestore resets or removes a listener target - after a reconnect, or when a stored resume token has expired - the cache drops what it holds for that collection and Firestore replays it. While that replay is in progress the collection stops answering list and query from the cache: read_through_cache falls back to Firestore, and read_cached_only returns a CacheError rather than a listing that looks complete but is not.

Full examples are available here and here.

Google authentication

Looks for credentials in the following places, preferring the first location found:

• A JSON file whose path is specified by the GOOGLE_APPLICATION_CREDENTIALS environment variable.
• A JSON file in a location known to the gcloud command-line tool using gcloud auth application-default login.
• On Google Compute Engine, it fetches credentials from the metadata server.

Local development

Don’t confuse gcloud auth login with gcloud auth application-default login for local development, since the first authorize only gcloud tool to access the Cloud Platform.

The latter obtains user access credentials via a web flow and puts them in the well-known location for Application Default Credentials (ADC). This command is useful when you are developing code that would normally use a service account but need to run the code in a local development environment where it’s easier to provide user credentials. So to work for local development you need to use gcloud auth application-default login.

Firestore emulator

To work with the Google Firestore emulator you can use the environment variable:

export FIRESTORE_EMULATOR_HOST="localhost:8080"

or specify it as an option using FirestoreDb::with_options().

When FIRESTORE_EMULATOR_HOST is set, the library does not look up the Google credentials and uses a stub token instead, since the emulator does not authenticate the requests. This means you do not need any credentials configured to develop against it. Specifying a token source explicitly, with FirestoreDb::with_options_token_source() for example, still takes precedence.

Docker images and TLS

When you design your Dockerfile make sure you either installed Root CA certificates or use base images that already include them. If you don’t have certs installed you usually observe the errors such as:

SystemError(FirestoreSystemError { public: FirestoreErrorPublicGenericDetails { code: "GrpcStatus(tonic::transport::Error(Transport, hyper::Error(Connect, Custom { kind: InvalidData, error: InvalidCertificateData(\"invalid peer certificate: UnknownIssuer\") })))" }, message: "GCloud system error: Tonic/gRPC error: transport error" })

For example for Debian based images, this usually can be fixed using this package:

RUN apt-get install -y ca-certificates

Also, I recommend considering using Google Distroless images since they are secure, already include Root CA certs, and are optimised for size.

TLS features

Cargo provides support for different TLS features for dependencies:

• tls-roots: default feature to support native TLS roots
• tls-webpki-roots: feature to switch to webpki crate roots

Request tags

Firestore supports attaching request tags to requests. They are reported by Firestore in its monitoring and billing breakdowns, which makes them useful to attribute reads and writes to a specific feature, tenant or background job.

Tags can be set per operation for queries, aggregations, listings and listeners:

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { some_id: String }
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
db.fluent()
    .select()
    .from(TEST_COLLECTION_NAME)
    .request_tags(["nightly-report"])
    // or use request_options if you want to provide the options structure directly
    // .request_options(FirestoreRequestOptions::from_tags(["nightly-report"]))
    .obj::<MyTestStructure>()
    .query()
    .await?;
Ok(())
}

Or session wide, for every request issued through a client instance. This is also how you attach tags to the CRUD operations (insert/update/delete/get):

use firestore::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { some_id: String }
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb, my_struct: MyTestStructure) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let tagged_db = db.clone_with_request_tags(["nightly-report"]);

tagged_db
    .fluent()
    .insert()
    .into(TEST_COLLECTION_NAME)
    .document_id(&my_struct.some_id)
    .object(&my_struct)
    .execute::<MyTestStructure>()
    .await?;
Ok(())
}

Transactions and batch writers accept them through their options:

use firestore::*;
use futures::FutureExt;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct MyTestStructure { some_id: String }
const TEST_COLLECTION_NAME: &str = "test";
async fn example(db: FirestoreDb) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
db.run_transaction_with_options(
    |db, _transaction| {
        async move {
            db.fluent()
                .select()
                .by_id_in(TEST_COLLECTION_NAME)
                .obj::<MyTestStructure>()
                .one("test-1")
                .await?;

            Ok(())
        }
        .boxed()
    },
    FirestoreTransactionOptions::new()
        .with_request_options(FirestoreRequestOptions::from_tags(["checkout"])),
)
.await?;
Ok(())
}

A per operation value replaces the session wide default rather than merging with it.

Testing your own code

The Fluent API is built on FirestoreDb directly, so there is no built-in way to substitute a fake database into db.fluent(). There are two approaches that work well.

Abstract at your own boundary

Define a trait for what your code needs and implement it over a type holding a FirestoreDb. Your business logic then depends on your trait, and tests provide their own implementation. This keeps Firestore concerns in one place and needs nothing special from this library:

use firestore::*;
use firestore::errors::FirestoreError;
#[derive(Debug, serde::Deserialize)]
struct User;
#[derive(Debug)]
struct MyError;
impl From<FirestoreError> for MyError {
    fn from(_: FirestoreError) -> Self {
        MyError
    }
}
#[async_trait::async_trait]
trait UserRepository {
    async fn find_user(&self, id: &str) -> Result<Option<User>, MyError>;
}

struct FirestoreUserRepository {
    db: FirestoreDb,
}

#[async_trait::async_trait]
impl UserRepository for FirestoreUserRepository {
    async fn find_user(&self, id: &str) -> Result<Option<User>, MyError> {
        Ok(self
            .db
            .fluent()
            .select()
            .by_id_in("users")
            .obj()
            .one(id)
            .await?)
    }
}

// In tests, implement `UserRepository` with an in-memory HashMap.

Run against the Firestore emulator

For tests that should exercise real query, listener and transaction behaviour, point the library at the Firestore emulator:

export FIRESTORE_EMULATOR_HOST="localhost:8080"

No credentials are needed in that mode. This is how the caching and transaction behaviour of this library itself is verified, and it catches things a hand-written fake cannot.

Changed in 0.52: the low level *Support traits are no longer public, so code written to be generic over them no longer compiles. Note that they could never be used with the Fluent API from outside the crate anyway, since the builders’ constructors are crate private. See the migration guide.