With a huge amount of labelled training data, multiple layers of non-linear features can be learnt by backpropagating error derivatives through a feedforward neural network. However, when very few labeled examples are available, the restricted Boltzmann machine (RBM), can be used to overcome the need for labeled data. several different generative models were developed that learned interesting features by modeling the higher order statistical structure of a set of input vectors. This study uses a RBMs to initialize the weights of a feedforward neural network that allows backpropagation to work effectively in deep networks and leads to good generalization. Using a stack of RBMs to initialize a deep Boltzmann machine with many hidden layers and combining with a new method for fine-tuning is shown to lead to the efficient training of Boltzmann machines with many hidden layers and millions of weights.