Yoshimune really wants to attend a festival in Edo, but he has important business to attend to in the palace. Jii says he can go out as long as he is back by 4 PM. This doesn't seem likely, since Yoshimune gets involved in a big fistfight and village drama.
