`w_in` in CubaLIF restricted to elementwise

[Jegp]

The current type check for w_in in CubaLIF currently restricts it to map 1:1 to the neurons, rather than permit a connection weight matrix.

According to the specs the CubaLIF is basically a LI -> Affine -> LIF,

so it should permit any Affine transformation.

Any opinions on updating the types to support connection matrices?

[benkroehs]

I may not understand this correctly but isn't w_in the weight applied on the input of the LI-part, not the affine transformation in between?

Now that I'm thinking about this: why is there an affine transformation in between the LI and LIF? Isn't there just a scaling of inputs by the resistance $R$? From my understanding, the LI models the current while the LIF models the voltage.

$\tau_m \frac{du}{dt} = -\left[u(t) - u_{\mathrm{rest}}\right] + RI(t)$

equation from https://neuronaldynamics.epfl.ch/online/Ch1.S3.html

[Jegp]

Thanks for the quick comment @benkroehs. Thanks for pointing this out! You're right that the w_in is the weights for the inputs. Which confused me because the r term in the CubaLIF is reserved for the second, membrane, term.
So, w_in is elementwise, which I reckon makes sense because I've noticed r is conventionally elementwise. But maybe we should consider renaming w_in to r_syn or similar...

To your question: we added it to permit linear combinations of synaptic "compartments". It's not a standard thing, though. And I recall we had a bit of hassle mapping it to all platforms (we mostly just error out if r != 1). One option would be to opt for removing it. That would also solve the above confusion.

My original question was about the r variable in the CubaLIF model. If we opt for a linear map from the synaptic to the somatic parts, I would think it makes sense to permit arbitrary connectivity. But it's a luxury problem; I could just define the graph I showed above. In any case, I'd prefer to disambiguate if we can.

[benkroehs]

I also do not have strong opinions about renaming w_in or not. I think leaving it as is is also fine since it really is just a scaling on the input.

About r: I've also seen the cases where r != 1 and i did the same thing for jaxsnn. But for hxtorch we need the membrane capacitance $C = \tau_m / R$. So I personally would like to keep it in some way. To extend the current implementation we could also allow a Linear node for R (similar to the reset case #138). Then we would have already two of these nested nodes. I think this nesting might be great way for expanding NIR in general.

[Jegp]

Nesting is an interesting idea. So, we would basically change the type of r and w_in to be Node, which can either be a Linear or Scale or something third? I like that, and, as you say, it's a nice way to expand NIR.

The downside is that the parsing will be trickier. Is this a problem for jaxsnn?

Also, what do you think about renaming the parameters to something like r_syn and r_mem? We should, of course, retain backwards compatibility. That should be quick wish a simple @property getter.

[benkroehs]

Yes exactly, I had this in mind. About the parsing: It is true that the introduction of such nested nodes would make things mode difficult because not all frameworks might support all sorts of nesting. But from my perspective, frameworks should simply assert that a (nested) node is of a supported type, regardless of whether it is nested or not.

So since jaxsnn just supports the scaling by 1, I would need to assert that the scaling parameter of the scaling node is 1

I think I would drop w_in in the future. We already default to 1, many frameworks do not support this in a clean way and in my opinion, such scaling should be done by a separate Linear/Scale node. We could then also simplify the notation by keeping the membrane resistance as r, since it would no longer be ambiguous.

The more I think about it, the more I feel that this is really about the direction NIR is evolving in and what we ultimately want NIR to look like — for example, whether we prefer a larger variety of node types or fewer, more complex nodes.