
The parameters, or synaptic weights as they are also called, are always
stored in the first element of a network, as described in 3.2.3: Network Format. The meaning of the individual
parameters is better explained in 13.1: Change the Parameter Values of an Existing Network.
Most neural networks can be evaluated on a symbolic input vector; then
the meaning of the parameters can be illustrated as in the example below.
Load the package.
Generate a network for demonstration purposes.
The numerical values of the parameters are all placed in the first element
of the network.
You can extract the first element and perform whatever process you want with
Mathematica.
An easy way to identify the meaning of the individual parameter is obtained by
applying the network to a symbolic input.
If you need to implement the trained network in, for example, a C program,
you might find the following command useful.
0.36758202492667336 + 0.8268122850621071*
Tanh(0.9682476245672822 + 0.11994745409360652*x1 
0.9841928909337702*x2) 
0.17743436966754977*Tanh(0.1432051850856072 
0.866500737817343*x1 + 0.8112145157992191*x2)


For most types of neural networks you concatenate the data sets into one set
before the training. This is shown in the first example below. For dynamic
neural networks, neural ARX and neural AR, it is slightly more complicated
because the individual data items are correlated. How you manage in this case
is shown in the second example.
Nondynamic Neural Networks
The general data format is described in 3.2.1: Data Format.
Load the package.
Generate a network for demonstration purposes.
In this data file, the input and output matrices are assigned to the variable
names x and y respectively. Once the data set has been loaded,
you can query the data using Mathematica commands. To better
understand the data format and variable name assignment, you may also want
to open the data file itself.
Load a data file containing input data x and output data y.
Show the contents of the input and output matrices.
Check the number of data items and the number of inputs and outputs for each.
There are 20 data samples; the input has one dimension, and the output has
two. Introduce a second data set.
The dimensionality of the input and output must be the same in the two data
sets.
A new, joined data set can now be constructed by concatenating the two input
data sets, and similarly with the output data sets.
Now you can start training your network with the combined data set.
Dynamic Neural Networks
Data from dynamic systems are correlated, and if two data sets are
concatenated in the way described in the previous example, then a transient
is introduced in the data. This transient might be small and unimportant.
There is, however, a way to avoid the transient totally. What you have to
do is to build the regressor, described in 2.6: Dynamic Neural Networks, for the two data sets and
then concatenate the result.
Load some packages.
<<NeuralNetworks`
<<Statistics`ContinuousDistributions`
<<Graphics`MultipleListPlot`
For demonstration purposes, generate a first data set.
Ndata = 30;
u1 = RandomArray[NormalDistribution[0, 3], {Ndata, 2}];
x = FoldList[Function[{xs, uin}, {(uin[[1]] + uin[[2]] +
0.6 * xs[[1]] + 0.8 * xs[[2]])/(1 + xs[[3]]^2), 0.7 * xs[[2]] +
uin[[2]], xs[[1]]}], {0, 0, 5}, Drop[u1, 1]];
y1 = x[[All, {1, 2}]];
Check dimensions of input and output, and the number of data for the first
data set.
Generate a second data set.
Ndata = 20;
u2 = RandomArray[NormalDistribution[0, 3], {Ndata, 2}];
x = FoldList[Function[{xs, uin}, {(uin[[1]] + uin[[2]] +
0.6 * xs[[1]] + 0.8 * xs[[2]])/(1 + xs[[3]]^2), 0.7 * xs[[2]] + uin[[2]], xs[[1]]}], {0, 0, 5}, Drop[u2, 1]];
y2 = x[[All, {1, 2}]];
Check dimensions of input and output, and the number of data for the second
data set.
Choose the indices indicating the regressor, as described in
2.6: Dynamic Neural Networks.
With one data set you can train a neural ARX model in the following way.
With two data sets you have to do the following.
Generate the regressors and modified outputs of the two data sets.
Concatenate regressors and the modified outputs in the same way as in the
previous example.
Initialize a neural ARX model of your choice (by indicating nil training
iterations in NeuralARXFit).
Extract the feedforward or the RBF network in the first position of the
initialized neural ARX model (as described in
8.1.1: Initializing and Training Dynamic Neural Networks).
Train the feedforward or the RBF network.
Put back the trained network at the first position of the neural ARX model.
You have now obtained a neural ARX model trained on both data sets. The
neural ARX model can be used, for example, to simulate the behavior of one
of the data sets.
