I remember I was able to reproduce your problem with not being able to
run Virtualbox as non-root user without window manager with older versions of Virtualbox
but it works for me with 5.2.4-119785.
However, as stated in the comments you don't need to graphical interface at all. Virtualbox comes with command line interface, it's my preferred method of interacting with it. Say, to list virtual machines:
$ vboxmanage list vms
"ubuntu" {e7fe8912-5484-4b9d-b5fe-431ae648b2dd}
"slackware64-current" {a00e1925-9cb6-4330-8b04-bbce69b49c67}
"slackware-current" {636b85f2-1e02-497b-9c50-22eb285250a3}
"freebsd" {acb77ca6-b051-400a-92bb-b3b2f1905991}
"openindiana" {bdca01dd-ea5d-4323-9fd2-92665d311bda}
"slackware64-14.1" {f9db3ba6-2316-4f92-8264-4abc8e4f71b3}
"slackware-14.2" {ccb5ddba-4ea1-4469-8ebc-21c84b8ca825}
"slackware-14.1-pathes-test" {67bec56d-b7d5-4427-a726-de2b4c1ba700}
"Slackware_new_test" {2e9232b5-bab1-41fc-8db4-ff4aab56f94c}
"slackware64-14.2" {2084cd19-d286-48b5-8e7b-3d1bb7a94d93}
"alpine linux" {be56c6a7-5821-4815-984b-6b40a6367acb}
"openwrt_Chaos_Calmer" {a4fcb438-dfc7-4672-8976-0d21b34357e0}
"linux-mint" {6e5e99d8-2dd8-4798-93ac-ffbf14c6c9db}
To start a virtual machine:
$ vboxmanage startvm "slackware64-14.1" --type headless
You can then access a started virtual machine with ssh, telnet etc.
To remove a virtual machine:
$ vboxmanage unregistervm "slackware64-14.1" -delete
To show info on a given virtual machine:
$ vboxmanage showvminfo slackware64-14.1
To stop a virtual machine:
$ vboxmanage controlvm slackware64-14.1 poweroff
You can even create a new virtual machine in command line: https://gist.github.com/ardrabczyk/65b68d0121f2964cd99e
VBoxHeadless --startvm vmNameshould do the trick. (... come to think of it, a--startvm WhatVMDoYouWantToStartargument is probably the missing ingredient.)xinit /usr/bin/VirtualBox --startvm "vm name" --fullscreen -- :0 vt1