Firestore Queries In Java

Hey Guys, let’s dig into Firestore queries in java. I suggest you to read our introductory article to Firestore with Spring Boot before follow along. As we discussed in previous article Firestore is a No SQL Database, and it doesn’t have query language. So how do we query? It’s easy, with the SDK they provide. In earlier article we’ve setup the project so let’s start with querying.

Why we don’t use spring cloud gcp firestore data?

We are not ready yet. I think its better to know what is underneath before do black magic. There is spring cloud gcp firestore data project and starter for spring boot. I wasn’t able to find matured documentation yet, but we could explore it when time comes. we will be able to use our favorite data repository pattern with Firestore. Until then let’s learn the fundamentals.

Firestore Indexes

I think you are familiar with indexes, are you? If not it’s time to learn the basics, find a good resource, take a cup of coffee and come back here after you done. No? Okay, every database optimize it’s performance by using indexes. Its same as we use a book index to locate the content. So if there is no index, database has to go through all the entries and find the matching entry for the query. Firestore uses indexes for every query it executes. So performance lies in the size of data return rather how many items are in the database. And there is good news for us, as developers we no longer need to manually handle the indexes as in conventional databases. Firestore creates required indexes for us for the basic queries and it generates error messages that helps us to create additional indexes it requires.

In last article, I didn’t talk about Firestore limitations and pricing. Cloud Firestore comes with various limitations, I would rather say restrictions to optimize the solution. And they charge if you exceeds the free quota. So how can we keep our budget low? we need to understand every aspect of Firestore to design optimal data model for the solution. Also managing indexes is a must, although it creates indexes for basic query. Let’s dig into Firestore indexes and data modeling in separate articles.

Let’s store some data first

First of all we need to have some data. I’ve done some changes to User class we created in previous article. Let’s save some users.

package com.aptkode.example.firestore;

import java.util.List;

public class User {
    private String name;
    private int age;
    private Gender gender;
    private List<String> interests;

    public User() {
    }

    public User(String name, int age, Gender gender, List<String> interests) {
        this.name = name;
        this.age = age;
        this.gender = gender;
        this.interests = interests;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public Gender getGender() {
        return gender;
    }

    public void setGender(Gender gender) {
        this.gender = gender;
    }

    public List<String> getInterests() {
        return interests;
    }

    public void setInterests(List<String> interests) {
        this.interests = interests;
    }
}

I’ve add Gender enum to address different data type usage.

package com.aptkode.example.firestore;

public enum Gender {
    MALE, FEMALE
}

Following method shows how to save set of documents, in this case users. I’ve used ApiFutures class to save list of documents.

private void insertUsers() throws ExecutionException, InterruptedException {
    List<ApiFuture<WriteResult>> futures = new ArrayList<>();
    CollectionReference users = this.firestore.collection("users");
    futures.add(users.document("tom").set(
            new User("tom", 18, Gender.MALE, Arrays.asList("ESports", "Swimming"))
    ));
    futures.add(users.document("stella").set(
            new User("stella", 25, Gender.MALE, Arrays.asList("Embroidery", "Cooking", "Swimming"))
    ));
    futures.add(users.document("john").set(
            new User("john", 28, Gender.MALE, Arrays.asList("Programming", "Cricket"))
    ));
    // blocking get
    List<WriteResult> writeResults = ApiFutures.allAsList(futures).get();
    writeResults.forEach(r -> logger.info("Updated time: {}", r.getUpdateTime()));
}

Batch writes

There is another way of doing this, if you don’t have any read operations you could use batch write. Simply only .set() .update() or .delete(). But there is one caveat though, max operations count is 500 and you get charged for each operation (yes, that’s how they charge, per operation) and if you have some field transformations then you get charged for them also. We’ll take you through those pricing aspect in different article. Let’s use batch write to do the same thing we did earlier.

private void insertUsersBatch() throws ExecutionException, InterruptedException {
    WriteBatch batch = this.firestore.batch();
    CollectionReference users = this.firestore.collection("users");
    batch.set(users.document("tom"),
            new User("tom", 18, Gender.MALE, Arrays.asList("ESports", "Swimming")));
    batch.set(users.document("stella"),
            new User("stella", 25, Gender.MALE, Arrays.asList("Embroidery", "Cooking", "Swimming")));
    batch.set(users.document("john"),
            new User("john", 28, Gender.MALE, Arrays.asList("Programming", "Cricket")));
    ApiFuture<List<WriteResult>> commit = batch.commit();
    // blocking get
    commit.get().forEach(r -> logger.info("Updated time: {}", r.getUpdateTime()));
}

You may notice that we’ve used get() all the time, but we could use addListener method on the ApiFuture to get a callback once operation is done. I’ll show how we can query asynchronously next.

Let’s query

For java there are different methods prefixed with where to make the query on collection reference. Let’s do some.

private List<User> getUsersByGender(Gender gender) throws ExecutionException, InterruptedException {
    CollectionReference users = this.firestore.collection("users");
    ApiFuture<QuerySnapshot> future = users.whereEqualTo("gender", gender.name()).get();
    // blocking get
    QuerySnapshot queryDocumentSnapshots = future.get();
    return queryDocumentSnapshots.getDocuments()
            .stream()
            .map( d -> d.toObject(User.class))
            .collect(Collectors.toList());
}

Let’s do it asynchronously.

private void getUsersByGenderAsync(Gender gender, Consumer<List<User>> consumer) {
    CollectionReference users = this.firestore.collection("users");
    users.whereEqualTo("gender", gender.name()).addSnapshotListener((value, error) -> {
        if(value != null) {
            List<User> results = value.getDocuments()
                    .stream()
                    .map(d -> d.toObject(User.class))
                    .collect(Collectors.toList());
            consumer.accept(results);
        }
    });
}
getUsersByGenderAsync(Gender.MALE, users -> logger.info("male users: {}", users));

Following is the log excerpt from the application

Firestore query log for asynchronous approach
Result thread in synchronous and asynchronous queries

As you can see in asynchronous way we get results in different thread, you can use different executor if you want by passing it as the first argument of the addSnapshotListener. So here we just use equal operator to query some documents. Same as that we have different methods such as whereLessThan, whereGreaterThanOrEqualTo, whereArrayContains and so on.

Let’s focus on bit different ones. whereIn query allows us to filter by field to have one of given values, its similar to do OR operations on same field. One thing to remember that it supports only upto 10 values.

private void getUsersByAges(List<Integer> ages, Consumer<List<User>> consumer) {
    CollectionReference users = this.firestore.collection("users");
    users.whereIn("age", ages).addSnapshotListener((value, error) -> {
        if (value != null) {
            List<User> results = value.getDocuments()
                    .stream()
                    .map(d -> d.toObject(User.class))
                    .collect(Collectors.toList());
            consumer.accept(results);
        }
    });
}

Other one is whereArrayContainsAny, it checks for array field where any of given value matches and returns that document. One thing to note that results are de duplicated, meaning if field contains two of criteria values only one document is returned although its matched twice. Here also we have 10 values restriction, And this is similar to do whereArrayContains multiple times with OR operator.

private void getUsersByInterests(List<String> interests, Consumer<List<User>> consumer) {
    CollectionReference users = this.firestore.collection("users");
    users.whereArrayContainsAny("interests", interests).addSnapshotListener((value, error) -> {
        if (value != null) {
            List<User> results = value.getDocuments()
                    .stream()
                    .map(d -> d.toObject(User.class))
                    .collect(Collectors.toList());
            consumer.accept(results);
        }
    });
}

Compound Queries

I’m going to tell you few bad things. As developers we need to query on multiple fields with different range filters, unfortunately in Firestore we only can combine range queries only on one field. Also if you are planing to query on same field using equality operator with range query or array contains you should have composite index. Following is example where I used two range filters.

private void getUsersInAgeRange(int minAge, int maxAge, Consumer<List<User>> consumer) {
    CollectionReference users = this.firestore.collection("users");
    users.whereGreaterThanOrEqualTo("age", minAge)
            .whereLessThanOrEqualTo("age", maxAge)
            .addSnapshotListener((value, error) -> {
        if (value != null) {
            List<User> results = value.getDocuments()
                    .stream()
                    .map(d -> d.toObject(User.class))
                    .collect(Collectors.toList());
            consumer.accept(results);
        }
    });
}

Querying on sub collections

First of all we need to have sub collection. Let’s have Address, user may have one or more addresses. Following is the Address class.

package com.aptkode.example.firestore;

public class Address {
    public enum Type{
        BILLING, RESIDENCE
    }
    private String country;
    private String city;
    private String state;
    private String street;
    private Type type;

    public Address() {
        // required by firestore
    }

    public Address(String country, String city, String state, String street, Type type) {
        this.country = country;
        this.city = city;
        this.state = state;
        this.street = street;
        this.type = type;
    }

    public String getCountry() {
        return country;
    }

    public void setCountry(String country) {
        this.country = country;
    }

    public String getCity() {
        return city;
    }

    public void setCity(String city) {
        this.city = city;
    }

    public String getState() {
        return state;
    }

    public void setState(String state) {
        this.state = state;
    }

    public String getStreet() {
        return street;
    }

    public void setStreet(String street) {
        this.street = street;
    }

    public Type getType() {
        return type;
    }

    public void setType(Type type) {
        this.type = type;
    }
}

Let’s update our users’ addresses.

private void updateAddresses() throws ExecutionException, InterruptedException {
    CollectionReference users = this.firestore.collection("users");
    List<ApiFuture<WriteResult>> futures = new ArrayList<>();
    futures.add(users.document("tom")
            .collection("addresses").document(Address.Type.RESIDENCE.name())
            .set(new Address("USA", "Los Angeles", "CA", "York Street", Address.Type.RESIDENCE)));
    futures.add(users.document("john")
            .collection("addresses").document(Address.Type.RESIDENCE.name())
            .set(new Address("USA", "San Francisco", "CA", "San Street", Address.Type.RESIDENCE)));
    ApiFutures.allAsList(futures).get();
}

Let’s query users who is live in given city.

private void getUsersLiveInCity(String city, Consumer<List<User>> consumer) {
    Query users = this.firestore.collectionGroup("addresses")
            .whereEqualTo("city", city);
    users.addSnapshotListener((value, error) -> {
        if (value != null) {
            List<User> results = value.getDocuments()
                    .stream()
                    .map(d -> d.toObject(User.class))
                    .collect(Collectors.toList());
            consumer.accept(results);
        }
        if(error != null){
            logger.error("failed", error);
        }
    });
}

Have you tried it? Or before your try have you notice I’ve logged error object. Becuase this will give us an error. To query on sub collections we should have index matching to collection. It gives us expressive message, just click the link and you’ll be in Firestore indexes page. It will ask you to add exemption, click add. Now you are good to go, how awesome it is right?

Query Limitations

No solution is perfect. In cloud firestore we also have few query limitations. We don’t have not equal query, still it can be achieved through less than + greater than query. Then we don’t have like queries, but there are quite expensive hacks we can do if its really required. Then for OR operations we could have upto 10 combinations. Finally it doesn’t support combining equal queries with others and specifically array queries doesn’t go with each other. If I have missed anything, let me know.

Wrap it up

Horray, You have made it. We discussed how to do simple queries, compound queries, synchronous and asynchronous approaches at last we covered queries on sub collections, then created index for sub collections. It’s quite a lot, so be happy. Let’s meet with another interesting topic. Happy Coding.

Photo Credit: Photo by Vlad Bagacian from Pexels

Leave a Comment

Your email address will not be published. Required fields are marked *