Diddy's federal sex trafficking and racketeering trial is undeniably weighing heavily on his family ... but his children's ...