Fix: RCU unlock out of error path

On channel error, RCU was not unlocking the read side. Furthermore,
remove a check for a NULL session that was also not going through an RCU
unlock. Change it to an assert.

This also adds a channel subbuf size check when enabling a channel.

Signed-off-by: David Goulet <dgoulet@efficios.com>