Skip to content

Mongo NoSQL FIND ignores Query.setMaxResults() / setFirstResult() because skip/limit are applied after iterator() #2814

Description

@mjlyman

Describe the bug

On the MongoDB NoSQL adapter, JPA Query.setMaxResults() and Query.setFirstResult() are accepted and mapped onto the interaction spec (mongo.limit / mongo.skip), but they are not applied to the MongoDB find.

MongoDatabaseInteraction opens the cursor with FindIterable.iterator() before calling skip(), limit(), and batchSize(). The Mongo Java driver executes the find when iterator() is called and copies the current limit into the FindOperation at that moment. A later limit() only mutates the iterable; it does not change the already-opened cursor.

The adapter then walks that unlimited cursor and materializes every matching document into a MongoListRecord (ArrayList). On a large collection this can fail with:

java.lang.OutOfMemoryError: Required array length 2147483639 + 1204 is too large

instead of returning a page of results.

To Reproduce

  • EclipseLink version: 4.0.6 (org.eclipse.persistence + org.eclipse.persistence.nosql)
  • Java/JDK version: 17
  • JEE Server: WildFly / JBoss EAP 8 (also reproducible in a standalone RESOURCE_LOCAL SE test)
  • Database provider/version: MongoDB 3.6+
  • Driver: org.mongodb:mongo-java-driver:3.12.14 (legacy com.mongodb.MongoClient API used by the 4.0.6 adapter)

Entity:

import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.Id;
import org.bson.types.ObjectId;
import org.eclipse.persistence.nosql.annotations.DataFormatType;
import org.eclipse.persistence.nosql.annotations.Field;
import org.eclipse.persistence.nosql.annotations.NoSql;

@Entity
@NoSql(dataType = "Item", dataFormat = DataFormatType.MAPPED)
public class Item {

    @Id
    @GeneratedValue
    @Field(name = "_id")
    private ObjectId id;

    private String name;

    public Item() {
    }

    public Item(String name) {
        this.name = name;
    }

    public ObjectId getId() {
        return id;
    }

    public String getName() {
        return name;
    }
}

persistence.xml:

<persistence xmlns="https://jakarta.ee/xml/ns/persistence" version="3.0">
    <persistence-unit name="test-mongo-pu" transaction-type="RESOURCE_LOCAL">
        <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
        <class>Item</class>
        <properties>
            <property name="eclipselink.target-database"
                      value="org.eclipse.persistence.nosql.adapters.mongo.MongoPlatform"/>
            <property name="eclipselink.nosql.connection-spec"
                      value="org.eclipse.persistence.nosql.adapters.mongo.MongoConnectionSpec"/>
            <property name="eclipselink.nosql.property.mongo.host" value="localhost"/>
            <property name="eclipselink.nosql.property.mongo.port" value="27017"/>
            <property name="eclipselink.nosql.property.mongo.db" value="testdb"/>
        </properties>
    </persistence-unit>
</persistence>

Code:

EntityManagerFactory entityManagerFactory = Persistence.createEntityManagerFactory("test-mongo-pu");
EntityManager em = entityManagerFactory.createEntityManager();

em.getTransaction().begin();
for (int i = 0; i < 25; i++) {
    em.persist(new Item("item-" + i));
}
em.getTransaction().commit();

List<Item> page = em.createQuery("SELECT i FROM Item i", Item.class)
        .setFirstResult(0)
        .setMaxResults(10)
        .getResultList();

// page.size() is 25 (all documents), not 10
System.out.println(page.size());

em.close();
entityManagerFactory.close();

Expected behavior

page.size() is at most 10. MongoDB receives a find with limit: 10 (and skip when setFirstResult is used).

Additional context

MongoPlatform.buildCallFromStatement() already maps JPA pagination onto the interaction:

https://github.com/eclipse-ee4j/eclipselink/blob/4.0.6/foundation/org.eclipse.persistence.nosql/src/main/java/org/eclipse/persistence/nosql/adapters/mongo/MongoPlatform.java#L301-L308

if (readQuery.getFirstResult() > 0) {
    interaction.setProperty(SKIP, readQuery.getFirstResult());
}
if (readQuery.getMaxRows() > 0) {
    interaction.setProperty(LIMIT, readQuery.getMaxRows());
}

Those properties are then copied onto MongoInteractionSpec in buildInteractionSpec(). The values never reach the server because FIND applies them after the cursor is opened:

https://github.com/eclipse-ee4j/eclipselink/blob/4.0.6/foundation/org.eclipse.persistence.nosql/src/main/java/org/eclipse/persistence/internal/nosql/adapters/mongo/MongoDatabaseInteraction.java#L154-L179

FindIterable<Document> iterable = collection.find(object);
if (sort != null) {
    iterable.sort(sort);
}
MongoCursor<Document> cursor = iterable.iterator(); // find already sent with limit=0
try {
    if (mongoSpec.getSkip() > 0) {
        iterable.skip(mongoSpec.getSkip());          // too late
    }
    if (mongoSpec.getLimit() != 0) {
        iterable.limit(mongoSpec.getLimit());        // too late
    }
    if (mongoSpec.getBatchSize() != 0) {
        iterable.batchSize(mongoSpec.getBatchSize());
    }

Driver confirmation for mongo-java-driver 3.12.14:

Calling limit() / skip() / batchSize() after iterator() cannot change that cursor.

Suggested fix: apply sort, skip, limit, and batchSize on the FindIterable before iterator():

FindIterable<Document> iterable = collection.find(object);
if (sort != null) {
    iterable.sort(sort);
}
if (mongoSpec.getSkip() > 0) {
    iterable.skip(mongoSpec.getSkip());
}
if (mongoSpec.getLimit() != 0) {
    iterable.limit(mongoSpec.getLimit());
}
if (mongoSpec.getBatchSize() != 0) {
    iterable.batchSize(mongoSpec.getBatchSize());
}
try (MongoCursor<Document> cursor = iterable.iterator()) {
    // materialize results
}

A Maven JPA bug-test-case project can be generated from EclipseLink's archetype if a runnable attachment is needed:

mvn archetype:generate -DarchetypeGroupId=org.eclipse.persistence -DarchetypeArtifactId=org.eclipse.persistence.bug.jpa-archetype -DarchetypeVersion=5.0.0-SNAPSHOT -DgroupId=eclipselink.bug.testcase -DartifactId=jpa-testcase

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions