VR? No Buy! Get Me Mixed Reality!

MR_RD.jpg
"No VR, no buy" has become a staple of new sim racing release reviews. Even F1 22 now offers the, to some, elusive technology. But I'm really not about that. I'm in for the (hopefully) next hardware development. Mixed Reality.

Almost a year ago, Yannik Haustein asked the question: "What Do You Think Is The Next Big Hardware Development?"

And my answer is: "I don't know, but I wish it were Mixed Reality!"

So let's explore this mythical world.

Why We Need Mixed Reality​

Some of you might be asking: "But why?"

Well, there's 1 simple problem with VR for me: I cannot actually see what my hands are doing. I wanna see where my fingers are; which button I'm pressing on the wheel or button box (aka 2nd keyboard).

I want to see.

I want more immersion.

And I know it's possible. See it as a free business Idea. I just don't know if it's a good business idea.

How it Could Work​

Mixed Reality as a theoretical concept in sim racing has been around for a while now. I mean, just look at this video from 2016!


Now while some of you might know this channel already, if you don't, here's a quick run-down.

The channel owner, a German man by the name of Marcel Pfister, edits recordings of himself doing VR racing. All the while he sits surrounded by a green screen and later edits the VR surroundings into the video.

Now here's the deal: I want that. But REALLY working.

The theory is simple: Have a VR headset with cameras in front. The cameras just project the images they capture, excluding the green screen. That would be live projected with the interior of the car.

Sounds simple, right?

Why It's Not That Simple​

But it isn't. Every new development needs time and money. And the question would be, how big is the market for this?

And the, quite frankly, deflating truth of my thought process is:

Not very big, at least in the home-usage sector.

Commercially it could be used for driver training lessons or the like. So the potential users would be every vehicle driver.

But in the home sector, can you think of more examples of usage than, say, sim racing and Farming Simulator?

I can't. (But if you can, let us know in the comments, please.)

And a lack of possible monetary reimbursement automatically means a lack of investment. Sad, but what can you do?

What YOU Can Do​

Now let me be a bit selfish for a second. I know this is my wish and all, but if I have sparked your interest in mixed reality, just spread the word. If enough people wish for it, sooner or later, someone will invest R&D into this field.

And that could fulfil the wish (hopefully) not only I have. The (possibly) most immersive sim rig ever:
  • Motion Rig
  • Green Screen
  • Mixed Reality
But until then, would you be interested in a mixed reality environment like this? Let us know in the comments down below (and if you are, please share the word)!
About author
Julian Strasser
Motorsports and Maker-stuff enthusiast. Part time jack-of-all-trades. Owner of tracc.eu, a sim racing-related service provider and its racing community.

Comments

I definitely wouldn't want to see my surroundings overlapped when playing in VR. I pretty much like the real reality just like it is when not simming. When simming, I want to be fully immersed in the game, not to see some of my real gear, my hands and so on. Mixed reality is something cool for youtube clips and should probably stay like that. To me it doesn't add more realism, but takes away some.
 
Mixed reality with a green screen would not be possible for me, because I would like a matching steering wheel for every car I drive and damn those Ferrari replica wheels are expensive!
 
Assetto Corsa's CSP mod has actually recently added the ability to track hands and fingers, so you actually can see your own digits now with a predicted arm. We aren't at the stage where we see ourselves in the car with no gloves and sat there in jeans but its getting closer. But that is still Virtual Reality.
This.

Speaking racing, I'm rooting towards the "it's still VR, unless you actually drive a car while being in VR" (sounds a bit hazardous to me, though :confused: :D).
Other types of VR sims walking around in your predefined guardian boundary play area the word 'mixed' is more relevant IMO.

However, the world is getting closer speaking the immersion factor.

Last weekend I played around with the Quest 2's finger mapping.
And suddenly found myself in AC identifying my hands and fingers in VR, just with sim racing gloves attached. And driving out the Alfa Romeo 4C with the nice mirrors, I almost got chocked suddenly identifying "myself" with helmet in the mirrors, mapping my head movement in the mirrors! Didn't even play with that in CM, apart from activation the 'VR camera shake' option, I think (but don't think that should make any difference?), so really caught me off guard. A nice surprise :)

Also last weekend, my girls being away a couple of hours, I finally took the initiative using AC's free cam option to it's full in combo with the Quest 2 open Play Area Boundary option.
In order to max out VR boundaries I rearranged our 4x10m living room, moved every furnature and other obstacles. Then placed my simring in the center of the living room. Loading AC, just turning out a couple of meters from my garage.
THEN leaving my sim seat, walking around the pit area.

In some track mods it's a really delightful experience with so many details, just to walk around in VR and discover. So got some spy jobs in other team garages :D:inlove:

I don't nessecarily need mapping of my simrig.
Even though using 4-6 different seat positions for my NLR F-GT Lite flexible seat, my seat position being dependant on sim car. I know the excact positions of my T8HA H-shifter/Seq.-shifter, wheels and my T300RS-GT wheel buttons very well.

Just as when you drive in the traffic or at track days, I know my 'buttons' without any look, having my focus 100% on the drive ways.

But one thing I must admit. I need VR mapping of keyboard.
UNLESS I'm following my long term rig plan to install physical a button solution, suiting my flexible seat. I don't need VR mapping of rig buttons, they'll just spoil my VR vintage classic car experience.

VR hands mirroring is fine for me, but mostly in pit area and open world free cruising in huge track mods, parking at places taking a view of the landscapes, once in a while.

Entering the VR world myself mid April 2022 (effectively used ~40 days until now), acquiring the Quest/Meta 2 I've been more discovering than racing, trying to get the most of 3D VR out of my ~15-20 most commonly used new - and older sims, using varying tools.

And really surprised how long I've been able to go so far with even minimum required hardware*.

All in all I'm extremely happy of how far you are able to go with low end old and cheap PC hardware and a starter level €299 VR headset and even able to get old and ancient sims working quite flawless, just by scrutineering and seaching the Internet for tools** and coding.

*OK upgraded my RX 580 to an RX 6700 XT yesterday (got teased from my prime desired vendor with much below MSRP prize tag sale), making my system really VR friendly. I.e. now running Ryzen 5 5600 X/RX 6700 XT/32GB DDR4 3200MHz (which is mid-to-high end of today's common VR standards, I think?).

** Have to mention the Steam Virtual Desktop tool as an option when you are on low spec PC hardware. I even got my newly acquired laptop of Ryzen 5 4600H/GTX 1650Ti Mobile/8GB DDR4 3200Mhz work effortless at medium settings in most sims and even high settings deactivating any kind of MSAA. And vorpX, either as stand alone or together with VD or together with Opentrack as well as the various ChrewChief VR plug-ins should also be mentioned, when trying to get older and ancient sims fully 3D functional in VR.

Edit: using VD, fav. combo GTR2 PnG + Alfa Romeo 1300 GTA Junior WB racing mixed PnG grid @ Montagna di Pietra is a massive BLAST even keyboard driving from above mentioned laptop - and again using open world boundaries you are able to do a walk around the town, being a 'live' spectator, enjoying the fabulous engine sounds nearing you before they enter your eyes around the corner - just wow (everybody writing about 'which sim has the best sound modeling', but the mods are missed in action here, this is the best you can get - also alot of 60ies and 70ies AC mods are top notch, adding alot to the total immersion factor, especially being in VR!).

Ofcourse all this mingling is not necessary for modern VR dedicated sims and the use of UE4 engine (AMS2 and PC2) tops ACC and especially rF2 and R3E were directly dissapointing (had to make use of VD to improve graphics and raise in-VR resolution).

Upgrading my GFX to more up-to-date standards saves me alot of time fine tuning and scrutineering, leving room for my limited time of optimizing the immersion factor.

Hey, maybe my next investment will be simrig buttons and maybe a buttkicker :)
 
Last edited:
if I think of daily driving my car, I’d say I never watch my fingers/hands when reaching out for some buttons let’s say on the radio. Even the hands on the steering wheel and gear shift lever are more a peripheral impression. Focus of attention is on the road ahead and the happenings in the road around me and in the rear view mirrors.

To my mind comes a saying about TF drivers on the Notdschleife stating „I go through that corner with XYZ kph“: if you have time to watch your speedometer, you’re not going fast enough.

If you have time to watch your fingers‘ movements, you’re not going fast enough.
 
Staff
Premium
Nice to have, but 170 -180 degree FOV from VR would be my dream before any hand movement tracking stuff. Being able to check your farside mirror and still see ahead with peripheral vision would be amazing. Maybe it will require tech and game support for multiple screens for each eye, with the outer ones running a lower resolutions.

With VR I don't check mirrors or over the shoulder like I would IRL. I mostly rely on radar apps for any cars behind my front wheels.
 
I really don’t agree at all.
It’s not immersive to sit in a car with SimRig button boxes at all.

After a while in VR, you can find all your buttons by muscle memory. In cars that’s actually look and feel real. It will brake the illusion with a SimWheel and button boxes.

But then again, that’s me. Good for you if you get your Mixed reality.
 
Premium
Assetto Corsa's CSP mod has actually recently added the ability to track hands and fingers, so you actually can see your own digits now with a predicted arm. We aren't at the stage where we see ourselves in the car with no gloves and sat there in jeans but its getting closer. But that is still Virtual Reality.
VERY NICE!

Does it work with the Steering wheel only, or does it attempt to handle shifting?

It would be next level if it could interpolate between the various gears in game with the gears of your control.

I think that games tend to handle feet well enough for all that you see them.
 
VERY NICE!

Does it work with the Steering wheel only, or does it attempt to handle shifting?

It would be next level if it could interpolate between the various gears in game with the gears of your control.

I think that games tend to handle feet well enough for all that you see them.
Sure.
A little nice howto I just found (ofcourse after using quite some time last weekend looking in CM myself):

And the vid I had in fresh mind when trying out CM settings:

All kudos to the uploaders.
 
Last edited:
Premium
Mixed reality already exists. The hololens are mixed reality glasses. It shows 3D objects in the real environment. I've seen demonstrations of it and it looks really great. The hololens I have seen are very comfortable to wear. It is very light weight and well balanced on your head. It costs something in the neighborhood of €16000 euros, so outside the budget of the average sim racer. I also have not heard or seen if there is already race sim software that can work with a hololens. But technically it is already possible.

I do wonder what sim racing would be like in mixed reality with a hololens. I think the experience will actually be less. With VR glasses on, your brain is in the cockpit of a race car that moves at speed. How would that combine with the still environment you are really in? I get the problem that you can't see where your hands and fingers are. I think software that can track where your hands and fingers are and display that in VR gives a better experience. For me that is allowed with racing gloves on.
 
VR is dead especially with the current gpu prices. Wake me up when there is a G force simulator (No...im not reffering to a movement platform)
 
Premium
Very interesting article. I'm always curious about the VR counter-argument about not being able to see your actual hands in the game.

I use VR and only VR and I can see my hands on my wheel at all times, they're there.... right in front of me.

I tend to race older cars, so I don't really need lots of buttons, but I do have a simple button box for some of the more up-to-date stuff. It's right next to my wheel, I know where everything is by muscle memory (it's also laid out in alphabetical order, so that makes it easier)

So this brings me back to the main point I'm clearly not understanding, is it more that you cannot see your virtual hands coming off the wheel to scratch your nose? Or is it that you cannot track your paw when it attempts to click something on your generic button box that happens to be floating, completely detached and clipping through the cockpit of your indycar? ;)
 
Premium
@Julian Strasser One question I didnt think to ask you when I posted which is do you use VR?

I like most people I know using VR dont have an issue using our button boxes etc, it doesnt take too long to get used to it.

So when you say
Well, there's 1 simple problem with VR for me: I cannot actually see what my hands are doing.
Are you actually posing the question as a person who doesnt use VR but sees what you think are issues or someone that uses VR and has issues?

I took it for granted that you are a VR user but of course I often see people say things that they think would put them off VR without having tried it and the OP is ambiguous as to whether these are real issues for you or perceived issues with current VR.
 
Premium
@Julian Strasser One question I didnt think to ask you when I posted which is do you use VR?

I like most people I know using VR dont have an issue using our button boxes etc, it doesnt take too long to get used to it.

So when you say

Are you actually posing the question as a person who doesnt use VR but sees what you think are issues or someone that uses VR and has issues?

I took it for granted that you are a VR user but of course I often see people say things that they think would put them off VR without having tried it and the OP is ambiguous as to whether these are real issues for you or perceived issues with current VR.
I do not use VR for Sim Racing thanks to that reason. I tried, but for me not being able to see what my hands are doing is an immersion killer.

So yes, I have a VR headset. Yes, I tried Sim Racing with VR. No, I don't use it anymore.

Also my "button box" is a keyboard, so getting used to finding the buttons there would take ages. And I just don't have this kind of endurance :D
 
Premium
This business about wanting to see your own hands.......Go for a drive in a real car. Start driving quickly. Are you looking at your hands or the wheel? I hope not.

I'm happy with the completely enclosed vr world. I'm like a blind man and have learned where everything is. Once again though, do you look at your real gearstick when driving, do you look at your indicators? No, you just use them.
You don't look at them directly, but I also don't look at them directly.

But the important factor for me is "peripheral vision". You don't see just what you are looking at directly, you see everything around it as well.

And yes, you can learn where everything is. After some time it will be second nature and you will need less and less peripheral vision as you realise where all your things are, but another important factor for me is the immersion to see in your peripheral view what your hands are doing.
 

Latest News

Article information

Author
Julian Strasser
Article read time
2 min read
Views
24,632
Comments
78
Last update

Online or Offline racing?

  • 100% online racing

    Votes: 74 7.3%
  • 75% online 25% offline

    Votes: 106 10.4%
  • 50% online 50% offline

    Votes: 147 14.5%
  • 25% online 75% offline

    Votes: 279 27.4%
  • 100% offline racing

    Votes: 407 40.0%
  • Something else, explain in comment

    Votes: 4 0.4%
Back
Top