Currently there is no place in a user's settings/info that I can see and manage all the numbers the user is assigned to.

The only way now is to go to the Numbers page and search for the user, and then edit each number separately.

Only when signing is as the user I can see (see attached) all the numbers they are assigned to, but can't manage them (as the user is not the admin.)

In the users settings page there needs to be a way for admins to see all numbers assigned to them and easily (press an X, or check multiple and click Unassign, or something like this) remove numbers from being assigned to this user.