Check the Lux documentation

Oct 13, 2025 by talynt.org 94 views

Introduction

In this article, we will explore an alternative approach to declaring a neural network using the Lux library in Julia. This approach is inspired by the code provided by Chris, which aims to integrate the neural network with the Catalyst library. However, we will encounter some issues while trying to get this code to work.

The Code

Let's start by looking at the code that we are trying to get working:

using Lux, Random, StableRNGs, ComponentArrays, ModelingToolkitNeuralNets
import ModelingToolkit.t_nounits as t
import ModelingToolkit.D_nounits as D
rng = StableRNG(1111)
const NN = Lux.Chain(Lux.Dense(2, 5), Lux.Dense(5, 5), Lux.Dense(5, 5),
    Lux.Dense(5, 2))

p, st = Lux.setup(rng, NN)
ca = ComponentArray{Float64}(p)
using ModelingToolkit
@parameters p[1:length(ca)] = Vector(ca)
@parameters T::typeof(typeof(ca))=typeof(ca) [tunable = false]

@variables X(t)
eqs = [
  D(X) ~ ModelingToolkitNeuralNets.stateless_apply(NN, X, ModelingToolkitNeuralNets.lazyconvert(T, p))
]

This code defines a neural network using the Lux library and tries to apply it to a variable X using the stateless_apply function from the ModelingToolkitNeuralNets module.

The Error

However, when we run this code, we get the following error:

ERROR: type Term has no field layer_1.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] throw_no_field(::Val{:Term}, s::Symbol)
   @ Unityper ~/.julia/packages/Unityper/BiodR/src/compactify.jl:505
 [3] getproperty
   @ ~/.julia/packages/Unityper/BiodR/src/compactify.jl:310 [inlined]
 [4] macro expansion
   @ ~/.julia/packages/Lux/ptjU6/src/layers/containers.jl:0 [inlined]
 [5] applychain(layers::@NamedTuple{…}, x::Num, ps::SymbolicUtils.BasicSymbolic{…}, st::@NamedTuple{…})
   @ Lux ~/.julia/packages/Lux/ptjU6/src/layers/containers.jl:515
 [6] (::Chain{…})(x::Num, ps::SymbolicUtils.BasicSymbolic{…}, st::@NamedTuple{…})
   @ Lux ~/.julia/packages/Lux/ptjU6/src/layers/containers.jl:513
 [7] apply(model::Chain{…}, x::Num, ps::SymbolicUtils.BasicSymbolic{…}, st::@NamedTuple{…})
   @ LuxCore ~/.julia/packages/LuxCore/paKmr/src/LuxCore.jl:155
 [8] stateless_apply(model::Chain{@NamedTuple{…}, Nothing}, x::Num, ps::SymbolicUtils.BasicSymbolic{AbstractVector{…}})
   @ LuxCore ~/.julia/packages/LuxCore/paKmr/src/LuxCore.jl:166

This error message is not very informative, but it seems to indicate that there is a problem with the layer_1 field of the Term type.

The Initial Example

Before we dive into the code, let's take a look at the initial example that we had:

const NN = Lux.Chain(Lux.Dense(2, 5, rbf), Lux.Dense(5, 5, rbf), Lux.Dense(5, 5, rbf),
    Lux.Dense(5, 2))

However, we couldn't find where to fetch the rbf variable from, so we removed it.

Possible Solutions

There are a few possible solutions to this problem:

Check the Lux documentation: The error message suggests that there is a problem with the layer_1 field of the Term type. We should check the Lux documentation to see if there is any information about this field.
Check the ModelingToolkit documentation: The stateless_apply function is from the ModelingToolkitNeuralNets module. We should check the ModelingToolkit documentation to see if there is any information about this function.
Try a different approach: If the above solutions don't work, we could try a different approach to declaring the neural network.

Conclusion

In this article, we explored an alternative approach to declaring a neural network using the Lux library in Julia. However, we encountered some issues while trying to get this code to work. We discussed possible solutions to this problem, including checking the Lux documentation, checking the ModelingToolkit documentation, and trying a different approach. We hope that this article has been helpful in understanding the issues with declaring a neural network using the Lux library.

Side Note

In the initial example, we had a neural network with a rbf activation function. However, we couldn't find where to fetch the rbf variable from, so we removed it. This might be related to the issues we are experiencing with the code.

Future Work

In the future, we plan to continue exploring the Lux library and its documentation to see if we can find a solution to this problem. We also plan to try a different approach to declaring the neural network.

References

Lux documentation: https://lux.readthedocs.io/en/latest/
ModelingToolkit documentation: https://modelingtoolkit.org/stable/

Code

using Lux, Random, StableRNGs, ComponentArrays, ModelingToolkitNeuralNets
import ModelingToolkit.t_nounits as t
import ModelingToolkit.D_nounits as D
rng = StableRNG(1111)
const NN = Lux.Chain(Lux.Dense(2, 5), Lux.Dense(5, 5), Lux.Dense(5, 5),
    Lux.Dense(5, 2))

p, st = Lux.setup(rng, NN)
ca = ComponentArray{Float64}(p)
using ModelingToolkit
@parameters p[1:length(ca)] = Vector(ca)
@parameters T::typeof(typeof(ca))=typeof(ca) [tunable = false]

@variables X(t)
eqs = [
  D(X) ~ ModelingToolkitNeuralNets.stateless_apply(NN, X, ModelingToolkitNeuralNets.lazyconvert(T, p))
]
```<br/>
**Q&A: Trying to Get Alternative (to Docs) Declaration to Work**
===========================================================

**Q: What is the problem with the code?**
----------------------------------------

A: The code is trying to declare a neural network using the Lux library in Julia, but it is encountering an error when trying to apply the neural network to a variable `X` using the `stateless_apply` function from the `ModelingToolkitNeuralNets` module.

**Q: What is the error message?**
-------------------------------

A: The error message is:

ERROR: type Term has no field layer_1. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] throw_no_field(::ValTerm, s::Symbol) @ Unityper ~/.julia/packages/Unityper/BiodR/src/compactify.jl:505 [3] getproperty @ ~/.julia/packages/Unityper/BiodR/src/compactify.jl:310 [inlined] [4] macro expansion @ ~/.julia/packages/Lux/ptjU6/src/layers/containers.jl:0 [inlined] [5] applychain(layers::@NamedTuple…}, x:Num, ps::SymbolicUtils.BasicSymbolic{…, st::@NamedTuple…}) @ Lux ~/.julia/packages/Lux/ptjU6/src/layers/containers.jl515 [6] (::Chain{…)(x::Num, ps::SymbolicUtils.BasicSymbolic…}, st:@NamedTuple{…) @ Lux ~/.julia/packages/Lux/ptjU6/src/layers/containers.jl:513 [7] apply(model::Chain…}, x:Num, ps::SymbolicUtils.BasicSymbolic{…, st::@NamedTuple…}) @ LuxCore ~/.julia/packages/LuxCore/paKmr/src/LuxCore.jl155 [8] stateless_apply(model::Chain{@NamedTuple{…, Nothing}, x::Num, ps::SymbolicUtils.BasicSymbolic{AbstractVector{…}}) @ LuxCore ~/.julia/packages/LuxCore/paKmr/src/LuxCore.jl:166

**Q: What is the problem with the `layer_1` field of the `Term` type?**
-------------------------------------------------------------------

A: The error message suggests that there is a problem with the `layer_1` field of the `Term` type. However, the Lux documentation does not provide any information about this field.

**Q: What is the `stateless_apply` function?**
--------------------------------------------

A: The `stateless_apply` function is from the `ModelingToolkitNeuralNets` module. It is used to apply a neural network to a variable without modifying the network's state.

**Q: What is the `lazyconvert` function?**
-----------------------------------------

A: The `lazyconvert` function is from the `ModelingToolkitNeuralNets` module. It is used to convert a variable to a type that can be used by the neural network.

**Q: What is the `ComponentArray` type?**
-----------------------------------------

A: The `ComponentArray` type is a type of array that is used to store the parameters of a neural network.

**Q: What is the `@parameters` macro?**
--------------------------------------

A: The `@parameters` macro is used to declare the parameters of a neural network.

**Q: What is the `@variables` macro?**
--------------------------------------

A: The `@variables` macro is used to declare the variables of a neural network.

**Q: What is the `D` function?**
---------------------------

A: The `D` function is used to compute the derivative of a variable.

**Q: What is the `t` function?**
---------------------------

A: The `t` function is used to compute the type of a variable.

**Q: What is the `typeof` function?**
-----------------------------------

A: The `typeof` function is used to compute the type of a variable.

**Q: What is the `Vector` type?**
---------------------------

A: The `Vector` type is a type of array that is used to store a collection of values.

**Q: What is the `SymbolicUtils.BasicSymbolic` type?**
------------------------------------------------

A: The `SymbolicUtils.BasicSymbolic` type is a type of symbolic expression that is used to represent the parameters of a neural network.

**Q: What is the `@NamedTuple` macro?**
--------------------------------------

A: The `@NamedTuple` macro is used to declare a named tuple.

**Q: What is the `Lux.Chain` type?**
-----------------------------------

A: The `Lux.Chain` type is a type of neural network that is composed of multiple layers.

**Q: What is the `Lux.Dense` type?**
----------------------------------

A: The `Lux.Dense` type is a type of neural network layer that is used for dense connections.

**Q: What is the `rbf` activation function?**
-----------------------------------------

A: The `rbf` activation function is a type of activation function that is used in neural networks.

**Q: What is the `Lux.setup` function?**
--------------------------------------

A: The `Lux.setup` function is used to set up a neural network.

**Q: What is the `ComponentArray{Float64}` type?**
------------------------------------------------

A: The `ComponentArray{Float64}` type is a type of array that is used to store the parameters of a neural network.

**Q: What is the `ModelingToolkitNeuralNets.stateless_apply` function?**
-------------------------------------------------------------------

A: The `ModelingToolkitNeuralNets.stateless_apply` function is used to apply a neural network to a variable without modifying the network's state.

**Q: What is the `ModelingToolkitNeuralNets.lazyconvert` function?**
----------------------------------------------------------------

A: The `ModelingToolkitNeuralNets.lazyconvert` function is used to convert a variable to a type that can be used by the neural network.

**Q: What is the `ModelingToolkit.t_nounits` function?**
------------------------------------------------

A: The `ModelingToolkit.t_nounits` function is used to compute the type of a variable.

**Q: What is the `ModelingToolkit.D_nounits` function?**
------------------------------------------------

A: The `ModelingToolkit.D_nounits` function is used to compute the derivative of a variable.

**Q: What is the `StableRNG` type?**
----------------------------------

A: The `StableRNG` type is a type of random number generator that is used to generate random numbers.

**Q: What is the `rng` variable?**
------------------------------

A: The `rng` variable is a random number generator that is used to generate random numbers.

**Q: What is the `p` variable?**
---------------------------

A: The `p` variable is a vector of parameters that is used to represent the neural network.

**Q: What is the `st` variable?**
---------------------------

A: The `st` variable is a vector of state variables that is used to represent the neural network.

**Q: What is the `ca` variable?**
---------------------------

A: The `ca` variable is a vector of parameters that is used to represent the neural network.

**Q: What is the `eqs` variable?**
---------------------------

A: The `eqs` variable is a vector of equations that is used to represent the neural network.

**Q: What is the `X` variable?**
---------------------------

A: The `X` variable is a variable that is used to represent the input to the neural network.

**Q: What is the `D(X)` expression?**
----------------------------------

A: The `D(X)` expression is used to compute the derivative of the variable `X`.

**Q: What is the `ModelingToolkitNeuralNets.stateless_apply(NN, X, ModelingToolkitNeuralNets.lazyconvert(T, p))` expression?**
-------------------------------------------------------------------------------------------------------------------

A: The `ModelingToolkitNeuralNets.stateless_apply(NN, X, ModelingToolkitNeuralNets.lazyconvert(T, p))` expression is used to apply the neural network `NN` to the variable `X` without modifying the network's state.

**Q: What is the `ModelingToolkitNeuralNets.lazyconvert(T, p)` expression?**
------------------------------------------------------------------------

A: The `ModelingToolkitNeuralNets.lazyconvert(T, p)` expression is used to convert the variable `p` to a type that can be used by the neural network.

**Q: What is the `typeof(ca)` expression?**
-----------------------------------------

A: The `typeof(ca)` expression is used to compute the type of the variable `ca`.

**Q: What is the `typeof(ca)` expression?**
-----------------------------------------

A: The `typeof(ca)` expression is used to compute the type of the variable `ca`.

**Q: What is the `Vector(ca)` expression?**
-----------------------------------------

A: The `Vector(ca)` expression is used to create a vector from the variable `ca`.

**Q: What is the `@parameters p[1:length(ca)] = Vector(ca)` expression?**
-------------------------------------------------------------------

A: The `@parameters p[1:length(ca)] = Vector(ca)` expression is used to declare the parameters of the neural network.

**Q: What is the `@parameters T::typeof(typeof(ca))=typeof(ca) [tunable = false]` expression?**
---------------------------------------------------------------------------------------------

A: The `@parameters T::typeof(typeof(ca))=typeof(ca) [tunable = false]` expression is used to declare the type of the variable `T`.

**Q: What is the `@variables X(t