Online training narx

[q138ben]

Hi, the package looks very fantastic to predict time series data. I was wondering if it is possible to implement it in an online training.

[jxx123]

Can you elaborate on online training? Do you mean updating the model parameters at each time step as new data come in?

[q138ben]

Hi, The current NARX model needs be trained with the provided training sequence with the model.fit() method. Only after the training can the model be used in other new data. I was wondering if the model can be updated constantly with each time step as new data comes in? like you said during the training process. Best regards, Ben

…

________________________________ From: Jinyu Xie <[email protected]> Sent: 08 October 2020 17:32 To: jxx123/fireTS Cc: Binbin Su; Author Subject: Re: [jxx123/fireTS] Online training narx (#7) Can you elaborate on online training? Do you mean updating the model parameters at each time step as new data come in? - You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIIKVOKFP2YDBHBAPBY6C3DSJXEPZANCNFSM4SBNYVEQ>.

[jxx123]

I see. What you can do is calling fit(X[:t], y[:t]) (considering t is the current time step) at each time step, but this could be slow since you retrain the model at each time step.

I know there are very efficient way to do this online update for linear models (https://www.mathworks.com/help/ident/ref/recursivearx.html), but I don't think there are such efficient algorithm for nonlinear models.

[q138ben]

Hi, I have tried fit() method to train the model at each time step for 200 time steps for example?. But the resulting model is terrible that seems to learn nothing in this way. But with the 200 time steps as a whole for training, I can get a really good model. So I suspect that calling fit() method each time will automatically reinitialize the weights which makes this 'online training' totally void. Best regards, Ben

…

________________________________ From: Jinyu Xie <[email protected]> Sent: 08 October 2020 17:52 To: jxx123/fireTS Cc: Binbin Su; Author Subject: Re: [jxx123/fireTS] Online training narx (#7) I see. What you can do is calling fit(X[:t], y[:t]) (considering t is the current time step) at each time step, but this could be slow since you retrain the model at each time step. I know there are very efficient way to do this online update for linear models (https://www.mathworks.com/help/ident/ref/recursivearx.html), but I don't think there are such efficient algorithm for nonlinear models. - You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIIKVOJXLGGQRWG5IMTTRBLSJXG4HANCNFSM4SBNYVEQ>.

[jxx123]

Yeah, it is possible to reinitialize the weights (depending on the underlying model you are using, say RandomForest or any neural networks) each time you call fit, but is has nothing to do with the fireTS package (it is caused by the underlying model). You might be able to tweak around for deep neural networks by initializing the weight using the previously trained weights (like transfer learning), but not sure if other sklearn models support this custom initial weights or not.

On a high level, if all the machine learning packages have a good API for initializing weights, maybe I can do something in fireTS. I will leave this issue open and maybe add more details once I find out more.