In this follow up post we apply the same methods we developed previously to a different dataset. In this third post we mix the previous two datasets.
So, on the one hand, we have noise signals, on the other hand we have innovators and followers.
Again, as previously, we make use of the github hosted package timeseries_utils, and a script, which follows here:
# load data
import os
os.chdir('** use your own dictionary, where you want to save your plots **')
import numpy as np
import matplotlib.pyplot as plt
from timeseries_utils import Configuration, Data
from timeseries_utils import calculateForecastSkillScore
# configuration
C = Configuration()
C.dictionary['nOfPoints'] = 600
# below 300 things dont work so nicely, so we use a value considerably above that
C.dictionary['inSampleRatio'] = 0.5
C.dictionary['differencingOrder'] = 1
C.dictionary['p_order'] = 7
C.dictionary['steps_prediction'] = 1
C.dictionary['epochs'] = 100
C.dictionary['verbose'] = 0
C.dictionary['modelname'] = modelname
# create data
# # set random seed for reproducibility
np.random.seed(178)
D = Data()
nOfSeries = 7
artificial_x, artificial_SERIES = artificial_data(nOfPoints = C.dictionary['nOfPoints'],
nOfSeries = nOfSeries,
f_rauschFactor = 0.5)
D.setInnovator('innovator1', C=C)
D.setFollower(name = 'follower1', otherSeriesName='innovator1', shiftBy=-1)
w1 = 0.5
w2 = 1 - w1
f3 = w1 * D.dictionaryTimeseries['innovator1'] + w2 * artificial_SERIES[:,3]
f4 = D.dictionaryTimeseries['innovator1'] * artificial_SERIES[:,3]
f5 = np.cos(D.dictionaryTimeseries['innovator1']) * np.sin(artificial_SERIES[:,3])
D.setFollower('mixture1',
otherSeriesName='innovator1 + a signal',
otherSeries=f3,
shiftBy=-1)
D.setFollower('mixture2',
otherSeriesName='innovator1 * a signal',
otherSeries=f4,
shiftBy=-1)
D.setFollower('mixture3',
otherSeriesName='cos(innovator1) * sin(a signal)',
otherSeries=f5,
shiftBy=-1)
SERIES_train = D.train(D.SERIES(), Configuration=C)
SERIES_test = D.test(D.SERIES(), Configuration=C)
VARIABLES_train = D.train(D.VARIABLES(), Configuration=C)
VARIABLES_test = D.test(D.VARIABLES(), Configuration=C)
#define, fit model; predict with with model
Prediction, Model = defineFitPredict(C=C,
D=D,
VARIABLES_train=VARIABLES_train,
SERIES_train=SERIES_train,
VARIABLES_test=VARIABLES_test,
SERIES_test=SERIES_test)
# calculate Accuracy : 0% as good as NULL-Hypothesis, 100% is perfect prediction
Skill = calculateForecastSkillScore(actual=SERIES_test, predicted=Prediction, movingAverage=20)
f, axarr = plt.subplots(1, 2, sharey=False)
f.suptitle(predictionModel)
axarr[0].plot(SERIES_test)
axarr[0].plot(Prediction)
axarr[0].set_title('Series, Prediction vs Time')
axarr[0].set_xlabel('Time')
axarr[0].set_ylabel('Series, Prediction')
axarr[1].plot(SERIES_test, Prediction, '.')
axarr[1].set_title('Prediction vs Series')
axarr[1].set_xlabel('Series')
axarr[1].set_ylabel('Prediction')
plt.savefig('pictures/' + plotname + '_1.png', dpi=300)
plt.show()
f, axarr = plt.subplots(1, D.numberOfSeries(), sharey=True)
f.suptitle(predictionModel + ', Prediction vs Series')
seriesTitles = list(D.dictionaryTimeseries.keys())
for plotNumber in range(D.numberOfSeries()):
seriesNumber = plotNumber
axarr[plotNumber].plot(SERIES_test[:,seriesNumber], Prediction[:,seriesNumber], '.')
# axarr[plotNumber].plot(SERIES_test[:-x,seriesNumber], Prediction[x:,seriesNumber], '.')
if Skill[seriesNumber] < 50:
axarr[plotNumber].set_title('S. ' + str(Skill[seriesNumber]) + '%', color='red')
if Skill[seriesNumber] > 50:
axarr[plotNumber].set_title('S. ' + str(Skill[seriesNumber]) + '%', color='green', fontweight='bold')
axarr[plotNumber].set_xlabel(seriesTitles[seriesNumber])
axarr[plotNumber].set_ylabel('Prediction')
axarr[plotNumber].set_xticklabels([])
plt.savefig('pictures/' + plotname + '_2.png', dpi=300)
plt.show()
# end: # second set of plots
Data
As stated, the main difference to the previous posts is the data.
In this post the data is composed of an innovator, a follower and some mixtures.
Arima
As mentioned, the defineFitPredict needs to be defined for each forecasting technique. In the case of arima we use defineFitPredict_ARIMA, which is supplied by our package timeseries_utils.
from timeseries_utils import artificial_data
from timeseries_utils import defineFitPredict_ARIMA, defineFitPredict_DENSE, defineFitPredict_LSTM
defineFitPredict = defineFitPredict_ARIMA
predictionModel = 'Arima'
Now we run the script shown above.
The configuration is this:
The configuration is (in alphabetical order):
differencingOrder:1
epochs:100
inSampleRatio:0.5
...
nOfPoints:600
p_order:7
splitPoint:300
steps_prediction:1
verbose:0
The data summary is:
SERIES-shape:(600, 5)
follower1:{'shiftedBy': -1, 'follows': 'innovator1'}
innovator1:{'nOfPoints': 600}
mixture1:{'shiftedBy': -1, 'follows': 'innovator1 + a signal'}
mixture2:{'shiftedBy': -1, 'follows': 'innovator1 * a signal'}
mixture3:{'shiftedBy': -1, 'follows': 'cos(innovator1) * sin(a signal)'}
The result is:

Interpretation : As is evident in the figures and the reported skill scores arima struggles to predict anything meaningful at all, except for mixture1, which contains a signal. The more complicated signal-bearing mixtures muixture2 and mixture3 are not predicted any better than by the benchmark.
Dense
As mentioned, the defineFitPredict needs to be defined for each forecasting technique. In the case of our dense network we usedefineFitPredict_DENSE, which is also supplied by our package timeseries_utils.
from timeseries_utils import artificial_data
from timeseries_utils import defineFitPredict_ARIMA, defineFitPredict_DENSE, defineFitPredict_LSTM
defineFitPredict = defineFitPredict_DENSE
predictionModel = 'Dense'
modelname = predictionModel + '-mixture'
plotname = 'forecasting_3_dense'
Now we run the script shown above.
The model summary is this:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten_1 (Flatten) (None, 35) 0
_________________________________________________________________
dense_1 (Dense) (None, 32) 1152
_________________________________________________________________
dropout_1 (Dropout) (None, 32) 0
_________________________________________________________________
dense_2 (Dense) (None, 5) 165
=================================================================
Total params: 1,317
Trainable params: 1,317
Non-trainable params: 0
_________________________________________________________________
The result is:

Interpretation : Dense does a pretty good job. Of course it cannot predict the innovator1, however it noticeably fails with the complicated construction of mixture3.
LSTM
As mentioned, the defineFitPredict needs to be defined for each forecasting technique. In the case of our LSTM network we use defineFitPredict_LSTM, which is also supplied by our package timeseries_utils.
from timeseries_utils import artificial_data
from timeseries_utils import defineFitPredict_ARIMA, defineFitPredict_DENSE, defineFitPredict_LSTM
defineFitPredict = defineFitPredict_LSTM
predictionModel = 'LSTM'
Now we run the script shown above.
The model summary is this:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
lstm_1 (LSTM) (None, 16) 1408
_________________________________________________________________
dense_1 (Dense) (None, 5) 85
=================================================================
Total params: 1,493
Trainable params: 1,493
Non-trainable params: 0
_________________________________________________________________
The result is:

Interpretation : LSTM does a pretty good job. Of course it cannot predict the innovator1, in particular it noticeably fails with the complicated construction of mixture3.
| Method/Timeseries | Arima | Dense | LSTM |
|---|---|---|---|
| follower1 | ✕ | ✔ | ✔ |
| innovator1 | ✕ | ✕ | ✕ |
| mixture1 | ✔ | ✔ | ✔ |
| mixture2 | ✕ | ✔ | ✔ |
| mixture3 | ✕ | ✕ | ✕ |
The joint winners are Dense and LSTM.