Q: Can I use several data sets simultaneously to train a neural network?

For most types of neural networks you concatenate the data sets into one set before the training. This is shown in the first example below. For dynamic neural networks, neural ARX and neural AR, it is slightly more complicated because the individual data items are correlated. How you manage in this case is shown in the second example.

Nondynamic Neural Networks

The general data format is described in 3.2.1: Data Format.

Load the package.

<< NeuralNetworks`

Generate a network for demonstration purposes.

<< NeuralNetworks`

<< one2twodimfunc.dat;

In this data file, the input and output matrices are assigned to the variable names x and y respectively. Once the data set has been loaded, you can query the data using Mathematica commands. To better understand the data format and variable name assignment, you may also want to open the data file itself.

Load a data file containing input data x and output data y.

<< one2twodimfunc.dat;

Show the contents of the input and output matrices.

x

y

Check the number of data items and the number of inputs and outputs for each.

Dimensions[x]
Dimensions[y]

There are 20 data samples; the input has one dimension, and the output has two. Introduce a second data set.

x2 = {{3}, {4}};
y2 = {{2, 4}, {3, 5}};

The dimensionality of the input and output must be the same in the two data sets.

Dimensions[x2]
Dimensions[y2]

A new, joined data set can now be constructed by concatenating the two input data sets, and similarly with the output data sets.

xnew = Join[x, x2]

ynew = Join[y, y2]

Now you can start training your network with the combined data set.

Dynamic Neural Networks

Data from dynamic systems are correlated, and if two data sets are concatenated in the way described in the previous example, then a transient is introduced in the data. This transient might be small and unimportant. There is, however, a way to avoid the transient totally. What you have to do is to build the regressor, described in 2.6: Dynamic Neural Networks, for the two data sets and then concatenate the result.

Load some packages.

<< NeuralNetworks`

<<Statistics`ContinuousDistributions`
<<Graphics`MultipleListPlot`

For demonstration purposes, generate a first data set.

Ndata=30;
u1=RandomArray[NormalDistribution[0,3], {Ndata,2}];
x=FoldList[Function[{xs,uin},{(uin[[1]]+uin[[2]]+0.6*xs[[1]]+0.8*xs[[2]])/(1+xs[[3]]^2),0.7*xs[[2]]+uin[[2]],xs[[1]]}],{0,0,5}, Drop[u1,-1]];
y1=x[[All,{1,2}]];

Check dimensions of input and output, and the number of data for the first data set.

Dimensions[u1]
Dimensions[y1]

Generate a second data set.

Ndata=20;
u2=RandomArray[NormalDistribution[0,3], {Ndata,2}];
x=FoldList[Function[{xs,uin},{(uin[[1]]+uin[[2]]+0.6*xs[[1]]+0.8*xs[[2]])/(1+xs[[3]]^2),0.7*xs[[2]]+uin[[2]],xs[[1]]}],{0,0,5}, Drop[u2,-1]];
y2=x[[All,{1,2}]];

Check dimensions of input and output, and the number of data for the second data set.

Dimensions[u2]
Dimensions[y2]

Choose the indices indicating the regressor, as described in 2.6: Dynamic Neural Networks.

na = {2, 1};
nb = {1, 1};
nk = {1, 1};

With one data set you can train a neural ARX model in the following way.

{model1, fitrecord} = 
        NeuralARXFit[u1, y1, {na, nb, nk}, FeedForwardNet, {2}, 
          2];

With two data sets you have to do the following.

Generate the regressors and modified outputs of the two data sets.

{x1, yy1} = 
        MakeRegressor[u1, y1, {na, nb, nk}];
{x2, yy2} = MakeRegressor[u2, y2, {na, nb, nk}];

Concatenate regressors and the modified outputs in the same way as in the previous example.

x = Join[x1, x2];
y = Join[yy1, yy2];

Initialize a neural ARX model of your choice (by indicating nil training iterations in NeuralARXFit).

{model1, fitrecord} = 
        NeuralARXFit[u1, y1, {na, nb, nk}, FeedForwardNet, {2}, 0, 
          CriterionPlot -> False];

Extract the feedforward or the RBF network in the first position of the initialized neural ARX model (as described in 8.1.1: Initializing and Training Dynamic Neural Networks).

ffrwd = model1[[1]]

Train the feedforward or the RBF network.

{ffrwd, fitrecord} = NeuralFit[ffrwd, x, y, 5];

Put back the trained network at the first position of the neural ARX model.

model1[[1]] = ffrwd;

You have now obtained a neural ARX model trained on both data sets. The neural ARX model can be used, for example, to simulate the behavior of one of the data sets.

NetComparePlot[u1, y1, model1]