Ability to import stores at startup

[alexandrebrg]

Hi, for development purposes, it would be useful to have a way to import store(s) at startup. It would be an argument like run --import-stores command that will allow to import DSL* or json store (including authorization models...) files from a specific folder (e.g. /var/openfga/import/). Finally, in case the store already exists, it would ignore the import.

For instance, Keycloak does this by reading all the files in a specific folder (e.g. /opt/keycloak/data/import/), it imports compatible models.

*OpenFGA only accepts json format now, but we can imagine that it might be able in the future to transform into json on the fly

How would it improve developer experience ?

• When using development stacks like docker-compose, give ability to share volumes and make easier importing stores with sample data
• When deploying to in cloud infrastructures, it allows to make easier the initial import of stores (e.g. ConfigMaps as volumes in kubernetes deployment)
• For newcomers, it would make easier the "Getting Started"
• With further development, it would be able to import assertions and tuples to test your stores automatically in CI

[aaguiarz]

Hi @alexandrebrg

Interesting idea!

What is the use case you are most interested on?

[alexandrebrg]

Definitely the ability to import a store from json to our local development stack, when we create our local stack we want all services to be automatically provisioned. As a workaround, we created a simple tool to achieve export / import of stores easily. It is used along openfga migration to have an up-to-date database.

[DGuhr]

This answer is meant to continue the thread in openFGA discord here: https://discord.com/channels/759188666072825867/930524706854031421/1052247820318556170

@alexandrebrg @aaguiarz @rhamzeh trying not to write too much, I try to solve (in parts) the same use case using different methods, namely a testcontainers based, "TDD" approach to openFGA, that would allow us to explore models and write checks very easily and automated, and thus fasten the pace of development.

Current very heavy WIP v 0.0.1 v0.0.2 you can find here:

This test uses:

• a docker compose file to automatically provision postgres and openFGA
• a model.json file that gets automatically provisioned at startup (currently inside the test method, but easily extractable to say, a "setup" method or similar)
• integration of the client SDK so you could easily e.g. call the check endpoint inside the test and assert on the resulting response

Todo:

• also import tuples and assertions from json(done on latest branch version)
• write actual tests

Mid- to long term this should enable a test driven approach to openFGA when exploring models and changes to models at development time, with early and fast feedback, all without leaving the IDE for the developer. All they should need to do is write e.g. a test containing sth around the lines of:

openFgaContainer
    .WithDbImage("postgres:14")
    .WithStore("testStore")
    .WithModel("model.json")
    .WithTuples("tuples.json")
    .WithAssertions("assertions.json")
    .Start()
   ...
    openFgaContainer.runAssertions("testStore")
    .assertNoAssertionErrors()
    ...
    openFgaContainer.runTupleQuery("is user:anne related to organization:openfga as repo_writer")
    
    ...
    openFgaContainer.check("user:anne reader repo:openfga/openfga true")

etc etc - it's not there at all, given that I am very new to openFGA and golang itself. But it's a start. All just by running the test in your IDE of choice. Without "real" databases or openfgas to provision somewhere, all dockerized, so avoiding the "it works on my machine" (the reason docker was invented if you ask me ;) )

Missing parts I could imagine mid- to longterm:

• better IDE support for writing and visualizing policies, e.g. via intellij plugin, vscode plugin etc.
• better structure of the json that could be exported from playground (currently i have to manually extract what is in model.json, tuples.json etc)
• testSDK containing special assertions like the assertNoAssertionErrors mentioned above.
• testSDK that allows continuous testing similar to what quarkus does it in dev mode whenever you change something in the json files in the same directory (or configurable one)
• a UI for non tech-savvy users to check this out ;)

Let me stop here, and try to get step 1 working first. 😅 I Just think that would give me a good developer experience and fast feedback loops when evolving a model or trying a use case with an existing model, all without leaving my IDE, all automated and easily integratable in a CI/CD workflow. So... some value in it i guess ;)

[DGuhr]

Cleaned the example up a bit, and as sometimes a picture tells more than a thousand words, this is where I'm at so-far:

• dockerized openfga + postgres setup with automatic provisioning/deprovisioning inside the test.
• openfga gets filled using json files for model, tuples, assertions
• go clientSDK integrated, so now one could write actual tests against openfga using e.g. check etc.

[aaguiarz]

OK, makes sense, thanks @alexandrebrg, we added it to the project backlog here openfga/roadmap#12