I don't know why you couldn't, because I just tried it, first from the administrator account logging out a user account with admin privileges, then just the reverse, and it worked both times.
Yes, you can log off a user from Task Manager. The tutorial below can help show you how to. You have to click on Show processes from all users to run Task Manager as administrator first to be able to though.
Look at the note in step 3 in the tutorial link above. You will need to do this from within an administrator account, and click on the Show processes from all users button in Task Manager first.
My Computer
Computer type
PC/Desktop
Computer Manufacturer/Model Number
Self built custom
OS
64-bit Windows 11 Pro for Workstations
CPU
Intel i7-8700K OC'd to 5 GHz
Motherboard
ASUS ROG Maximus XI Formula Z390
Memory
64 GB (4x16GB) G.SKILL TridentZ RGB DDR4 3600 MHz
Graphics Card(s)
ASUS ROG-STRIX-GTX1080TI-O11G-GAMING
Sound Card
Integrated
Monitor(s) Displays
2 x Samsung Odyssey G7 27"
Screen Resolution
2560x1440
Hard Drives
1TB Samsung 990 PRO M.2,
4TB Samsung 990 PRO PRO M.2,
TerraMaster F8 SSD Plus NAS
PSU
Seasonic Prime Titanium 850W
Case
Thermaltake Core P3
Cooling
Corsair Hydro H115i
Keyboard
Logitech wireless K800
Mouse
Logitech MX Master 4
Internet Speed
2 Gb/s Download and 100 Mb/s Upload
Antivirus
Malwarebyte Anti-Malware Premium
Browser
Google Chrome
Other Info
Logitech Z625 speaker system,
Logitech BRIO 4K Pro webcam,
HP Color LaserJet Pro MFP M477fdn,
APC SMART-UPS RT 1000 XL - SURT1000XLI,
Galaxy S23 Plus phone