I agree with the nurse as well. I was lucky enough that i could log on at home and work from home a few days a week, but after weeks I was back in the office most days.
I would come back some nights at 6.30 pm having been away from home since 8.30 am and he hadn't woken up or eaten or left the bed. I had to be hard, tell him off and force feed him. I started calling him from work every 2 hours just to make sure he was awake and eating. It worked.
He needs to stop relying on you. You will be there for him when you come back from work, and he can always call you. You will feel terribly guilty, but you are being cruel to be kind. You don't want him to be stuck in a "oh poor me I am so helpless" phase. Part time as a start sounds excellent. He can get used to being not there. Plus as Christine said it will give you a boost being out there in the outside world.
Good luck