Can I give different colleagues access to different pieces of work for the same client?

Yes, you can. Add the correct colleagues to the appropriate work items by assigning them a task, the entire work item, @mentioning them in a note or assigning an email to them.