Put this in your .tmux.conf on both the local and the remote machines. You will have to edit it a bit if you have your keys mapped differently. Operation goes like this: press Shift-Up and you will be controlling the inner (remote) tmux, press Shift-Down to control the outer one.

source: L