Sequential vs. Functional

Keras Models: Sequential vs. Functional

There are two ways to build Keras models: sequential and functional.

The sequential API allows you to create models layer-by-layer for most problems. It is limited in that it does not allow you to create models that share layers or have multiple inputs or outputs.

Alternatively, the functional API allows you to create models that have a lot more flexibility as you can easily define models where layers connect to more than just the previous and next layers. 

Reference:
https://jovianlin.io/keras-models-sequential-vs-functional/

Teacher forcing

Teacher forcing is a strategy for training recurrent neural networks that uses model output from a prior time step as an input.

... the decoder learns to generate targets[t+1...] given targets[...t], conditioned on the input sequence.

https://machinelearningmastery.com/teacher-forcing-for-recurrent-neural-networks/

RMSprop

So that's RMSprop, and similar to momentum, has the effects of damping out the oscillations in gradient descent, in mini-batch gradient descent. And allowing you to maybe use a larger learning rate alpha. And certainly speeding up the learning speed of your algorithm.

from Andrew Ng's lecture
https://www.youtube.com/watch?v=_e-LFe_igno