How do I avoid materializing tensors of state transitions?

[meditans]

Hi, I recently started learning about active inference, and used the python pymdp library to start coding examples as I went along. However, I found a limitation, and I'd like to understand if I can get around it with RxInfer.

Imagine your state is a 10x10 grid, in which each cell can be on or off (independently), so the number of possible states is 2^100. We want to express the probability of observation given the state, but in pymdp this is encoded in a tensor, and this tensor will have n_observation x n_states elements. Given the big number of states, it's impossible to create this tensor if one wants to model large grids. But in my case that matrix is extremely sparse, and so I wondered if RxInfer had a way of encoding it in a more memory efficient way. I would like to write it down as just a function. As a data point, the pomdp control example uses the DiscreteTransition node, which seems to be encoding the information as a tensor.

Do you have any suggestion?

[bvdmitri]

I guess you can just use SparseArrays to encode the tensor. Sparse arrays are a part of Julia standard library. @wouterwln may have a better insight on whether the rules will be working efficiently with the sparse matrices or not, but as far as I know the rules do not require you to pass the tensor as a dense matrix so you can very well try the sparse matrices from the standard library

[wouterwln]

Hi @meditans , Thanks for trying out RxInfer! It just so depends on the dependencies in your generative model. If every observation can be influenced by every state, then I'm afraid you cannot get around the fact you will need a $2^{100}$ sized tensor. This is not a constraint of RxInfer or anything, but of your generative model. Where RxInfer shines is because you actually CAN reduce the size of this tensor, if we assume a structure of the model. Let's say, for example, that every observation is only influenced by the state it represents. Then we will have 100 $2 \times 2$ matrices, which greatly reduces the computational complexity of the model! This is indeed possible with the DiscreteTransition node. Then, you can assume a structure in the generative model of your states (for example, an Ising model like I did here) to make sure you model the conditional dependencies between the different grid cells. Then you will indirectly have a dependency between the different states and the different observations, but because of the conditional independence assumptions you make, the inference procedure is a lot cheaper to compute.

I won't contain any code in my answer because I do not know what your generative model looks like or which dependencies you would like to model, but we can have a discussion on what your requirements would be. For a general rule of thumb: reducing the complexity in your mathematical model directly corresponds to reducing the computational complexity of the inference, and in general (the goal is) that RxInfer is able to run inference in any discrete state model you specify! Did I put you in the right track with this, or do you need a pointer to additional resources?

[meditans]

Thank you both for answering, and @wouterwln you're right, I meant every observation is only influenced by the state it represents (so we have 100 observations). What I want to do for this example is creating an active inference agent that sees these 100 locations, initially with belief [0.5, 0.5] each, and has an action for each one. I know that free energy minimization will compel it to open each location in turn to gain knowledge about the content (I did this experiment in pymdp with 3 cells),

Given DiscreteTransition is the correct node, I'm afraid I'm not sure how to write the relationship between state and observation. The difficulty mainly arises from the fact that I don't have a mental model for which object DiscreteTransition is meant to be practically. Looking at the source, here is the definition, but I guess it's empty because the bulk of the behavior is defined by the macros below?

[wouterwln]

Let's say your states are one in 4, and your observations are 1 in 5. DiscreteTransition warrants you to create a (5, 4) sized tensor, which is normalized along the first dimension (sum to 1). That way, for example, in cell (3, 1), we can read what the probability will be of observing 3 in state 1. You can write something like observations[i, j] ~ DiscreteTransition(states[i, j], observation_matrix) for every (i,j) pair in your grid, for example. Then you can write any joint model on your states (and maybe include actions) in order to do valid inference. Indeed, the source code for DiscreteTransition can be a bit esoteric at times, but this is exactly the complexity that RxInfer tries to hide for you! Hopefully this nudges you in the right direction a bit.