VR? No Buy! Get Me Mixed Reality!

MR_RD.jpg
"No VR, no buy" has become a staple of new sim racing release reviews. Even F1 22 now offers the, to some, elusive technology. But I'm really not about that. I'm in for the (hopefully) next hardware development. Mixed Reality.

Almost a year ago, Yannik Haustein asked the question: "What Do You Think Is The Next Big Hardware Development?"

And my answer is: "I don't know, but I wish it were Mixed Reality!"

So let's explore this mythical world.

Why We Need Mixed Reality​

Some of you might be asking: "But why?"

Well, there's 1 simple problem with VR for me: I cannot actually see what my hands are doing. I wanna see where my fingers are; which button I'm pressing on the wheel or button box (aka 2nd keyboard).

I want to see.

I want more immersion.

And I know it's possible. See it as a free business Idea. I just don't know if it's a good business idea.

How it Could Work​

Mixed Reality as a theoretical concept in sim racing has been around for a while now. I mean, just look at this video from 2016!


Now while some of you might know this channel already, if you don't, here's a quick run-down.

The channel owner, a German man by the name of Marcel Pfister, edits recordings of himself doing VR racing. All the while he sits surrounded by a green screen and later edits the VR surroundings into the video.

Now here's the deal: I want that. But REALLY working.

The theory is simple: Have a VR headset with cameras in front. The cameras just project the images they capture, excluding the green screen. That would be live projected with the interior of the car.

Sounds simple, right?

Why It's Not That Simple​

But it isn't. Every new development needs time and money. And the question would be, how big is the market for this?

And the, quite frankly, deflating truth of my thought process is:

Not very big, at least in the home-usage sector.

Commercially it could be used for driver training lessons or the like. So the potential users would be every vehicle driver.

But in the home sector, can you think of more examples of usage than, say, sim racing and Farming Simulator?

I can't. (But if you can, let us know in the comments, please.)

And a lack of possible monetary reimbursement automatically means a lack of investment. Sad, but what can you do?

What YOU Can Do​

Now let me be a bit selfish for a second. I know this is my wish and all, but if I have sparked your interest in mixed reality, just spread the word. If enough people wish for it, sooner or later, someone will invest R&D into this field.

And that could fulfil the wish (hopefully) not only I have. The (possibly) most immersive sim rig ever:
  • Motion Rig
  • Green Screen
  • Mixed Reality
But until then, would you be interested in a mixed reality environment like this? Let us know in the comments down below (and if you are, please share the word)!
About author
Julian Strasser
Motorsports and Maker-stuff enthusiast. Part time jack-of-all-trades. Owner of tracc.eu, a sim racing-related service provider and its racing community.

Comments

Dont worry, this is coming soon, hopefully in the next few months, a few bugs to fix first before I even want to do a proper demo.
 
I'm 50 and amidst the midlife crisis. I needed that Porsche badly to compensate... futhermore, it replaced a 20 year old Z3 that is not fit anymore for tracktime.
OK sliding a bit off the track now (ok past the gravel into the fenches I admit).

But hey, just driven past the midway mark here, too :)
Though, haven't felt any crisis yet, just noticing a strange immersion of passing the landmarks 25-30-40-50 at a seemingly growing speed.

For me, it's not one or another.
I love immersion in simracing. I love track days in dedicated race cars, I love my bikes. I love my girls. And so on. And still catching up on new challenges, e.g. startet up golfing a couple of months ago after 38 years of hesitation.

Now you mention the last remnants of "pure ICE" and then more or less in same phrase a Cayman...but oh boy, despite 100% ICE, even the Germans' DNA is also being lost in kindergarten aids and paddlemania nonsense.

Speaking real world, I have far more joy going full throttle at twisty corners in a 1970 NSU Prinz 1200 than likewise pedal in modern supercars (of which I've tried quite a few over the years), dedicated track cars edging Nordschleife I have much more fun in caged cars set up for track purpose, with less power, most likely with H-stick and heel&toe.

I'm convinced visiting old virtues of 'original authenticity' once in a while - both IRL and VR simwise - prevents me from growing into an ol' bitter man.
And VR simwise I even have the opportinity for almost daily visits :inlove:
 
Last edited:
Thread officially hijacked!

New topics:
Mid life crisis, life balance and retirement.
Please accept my unreserved apologies :D

Back on track;

While re-reading the article, again I get a bit confused by the new (new to me) term 'mixed reality', but I interpret it as a fluid schism between the already well-cemented concept 'augmentet reality' and the world of VR.
As stated before, finger mapping is a fine gimmick adding to the immerson, but I don't need mapping of my simrig. I'm admiring the excellent modders' work and prefer the full VR experience of their work.
(only missing some kind of mapping of my keyboard, ofcourse only activated when I grab it and switched off in the moment I leave it on my side desk, but think I'll be switching to a button solution soon).
 
Last edited:
I've also seen guys building rigs as they are reaching retirement with the idea this is something they will enjoy spending their time on. So far I've not seen that work out.
Well working just splendid here! Especially VR racing classic race cars at historic and now abandoned tracks so excellent modelled does the trick for me.
But maybe I'm out of your scope. Working in my country, I have yet at least 22-23 years before I meet the requirements for a retirement.
 
Last edited:
BTW: Somewhat related to article title, and though been ~90% VR racer lately (or rather "VR investigator") I'm not part of the sect "No VR, no Buy".
As written before, there's plenty of tools getting non VR-dedicated sims on the bandwagon.
Though could be great getting my original GP2 discs fully 3D functional in my Quest 2, but until then I'm enjoying immensly getting the sim working splendid @flatscreen together with modern sim hardware.
 
How did you manage to get the quest 2 finger tracking into ac? I can't find an answer anywhere
Sorry, I missed your question.
Previously in this thread I linked to a howto vid on this subject - did you notice the vid and follow the instructions strictly?

Well, I didn't myself (discovered it after my own trial&error) but did something similar and it works together with my Quest 2. OK, I'll sum up a few notes of importance.

  • Quest 2: Settings -> Hands & Controllers -> activate Hand tracking.
  • Use the Virtual Destop Streamer. It is correct as stated in the vid, that this will not work with Oculus Link, i.e. neither with link cable nor Oculus Air Link! Yes, the Virtual Desktop Streamer is not for free, but almost (and every penny/dime worth it, IMO, both for performance reasons but also due to you now have a free moving head without annoying link cable attached). From PC, VD Options: (VD has to be running before activating it within your Quest 2) remember to activated touch input. Quite important here!
  • Content Manager, AC Video settings, rendering mode: OpenVR - this will startup the SteamVR when you load AC within CM. Tip. if you in-car is out of focus, use the Quest 2 left controller settings to activate the SteamVR reset view button. Ensure to have your head in track-ahead position (it will count down 3-2-1 and you have proper cockpit view again).
  • CSP: from within CM, upgrade to one of the newer versions (I'm running CSP 1.78 right now) and for CSP settings -> Mode Tweaks: VR (activate extension if it isn't already), deactivate 'Render controller directions as blue lines' and activate 'Move driver model hands with controllers' (this does the trick both speaking your wheel and if your using stick together with VR hand movement).

Oh wait, maybe I should add one advice:
If you've set up everything precisely as in the instruction vid, but you don't see your arms, then try either just wait for 15-20 secs - or (better) grab your controllers and move them around.
If Hand Tracking is activated in Quest 2 and touch input is activated in Virtual Desktop, you should be able to see your hands and fingers. moving with your controllers.
Then put the controllers down.

And voilá! You should now see your arms moving in front of your virtual cockpit :)

Edit: right now I'm having quite a laugh, activated hand/finger movement in VR while keyboard driving in my arm chair, comfortably reclined, using a dedicated plate for keyboard&mouse with an ergonomic pillow included underneath it, resting at my tighs. If you don't know what I'm writing about, try it yourself! :D
 
Last edited:
Short story that no one will read:

When I tried sim racing, VR development was a sky-rocket, compared to the stagnant world of sims. I was literally buying a new HMD with DOUBLE the specs each year. I was ambitious I wrote the same ideas mentioned in this article in an RD thread called Wishlist for AMS2.
Fast forward to 2022 and it makes me sad to see that people are still using the same products from that era because there are no break-through's (except for Quest2).

Nothing doubles in sim racing each year. Motion sims are still a dream and people are returning to 15 year-old software because the yearly simcades aren't becoming any more accurate or real with each new version.

I would say the only news since 2018 are: Fanatec making DD technology prices mainstream (CSLDD), and a brand-new sim after Assetto Corsa engine got old (Rennsport).

I think we live in different times than when the first sim invented force feedback or laser photogrammetry ..etc.
 
ah yes, make something even MORE complicated to set up than vr. vr gloves with all the finger tracking is a much easier problem to solve. we're pretty much already there.
 
My VR headset is collecting dust because of that. Meanwhile sim racing is possible, my flight sim with radio stacks, autopilot panel and 200 buttons is 100% impossible without see the real environment.
 
ah yes, make something even MORE complicated to set up than vr. vr gloves with all the finger tracking is a much easier problem to solve. we're pretty much already there.
Even the cheap Quest2 can do hands- and finger-tracking out of the box already… (no gloves needed)
 
Last edited:
Even the cheap Quest2 can do hands- and finger-tracking out of the box already… (no gloves needed)
Yes, extremely easy to activate finger-tracking.
And speaking AC very easy too using newest CM VR features (besides tracking of arms/hands/fingers also the quite cool feature 'head tracking' in mirrors included in more cars).
But after recent software upgrade another case letting Quest 2 understand you've actually deactivated the feature again :D
 
Last edited:
Short story that no one will read:

When I tried sim racing, VR development was a sky-rocket, compared to the stagnant world of sims. I was literally buying a new HMD with DOUBLE the specs each year. I was ambitious I wrote the same ideas mentioned in this article in an RD thread called Wishlist for AMS2.
Fast forward to 2022 and it makes me sad to see that people are still using the same products from that era because there are no break-through's (except for Quest2).

Nothing doubles in sim racing each year. Motion sims are still a dream and people are returning to 15 year-old software because the yearly simcades aren't becoming any more accurate or real with each new version.

I would say the only news since 2018 are: Fanatec making DD technology prices mainstream (CSLDD), and a brand-new sim after Assetto Corsa engine got old (Rennsport).

I think we live in different times than when the first sim invented force feedback or laser photogrammetry ..etc.
I agree with you with racing games not developing as much as a lot of us would have hoped (how many games re-use the same physics engine and then just make their own cars / "mods" for it while pumping out tons of cars & tracks). I also agree with you on how VR headsets have done the same lately however, I must say, there is an exception to that...

Pimax. They have continually pushed the consumer market for VR. They are continually adding more features and higher specs. Higher resolutions, higher pixel density, higher v.FOV, higher h.FOV, higher refresh rates, hand tracking, eye-tracking, etc. Heck, even on Pimax's recently released Sword controllers, they pushed the tech further than any one else, at least in terms of sensors:

"Pimax has mentioned using an unprecedented 26 [52 total] optical sensors for each Sword. SteamVR previously had a hard limit of 32 [total] sensors per processor, to use more you have to run an additional processor which just adds another layer of complication, cost, and increased power budget. Pimax also mentions using a higher specification IMU with a faster data capture rate."

On top of that, the Pimax Crystal which is supposed to come out at the end of this month and the Pimax 12K which is supposed to come out Q4 2022 are a another giant leap forward in VR tech. Pimax is already ahead of all headsets in the vast majority of categories, with the Crystal and 12K, they'll be even further ahead. Assuming those headsets do what they claim, I can't see any other company (at least ones we know of) coming close to their specs for the next 2 - 4 years.
 
Last edited:
Premium
Based on history I have to take EVERYTHING Pimax announces with a cargo ship sized chunk of rock salt.

I would never bet on them shipping a product on time or on that product living up to the hype.

If they actually had a track record of delivering on time, then I might be more optimistic.

As it is I'm waiting patiently for Valve to drop their next VR headset that I'm sure I'll be happy with. They've earned loyalty with an excellent product.

If the new Pimax is "all that", I'll wait to hear it from a number of other people before I would consider ordering one.
 
Last edited:
Even if it doesn't come on time, it'll almost certainly still come. And even if it doesn't, the Pimax 8K X still blows every thing out of the water. And, aside from pixel density, so does the 5K Super.
 
I developed this some years ago already. Sadly its a terrible business idea and it isn't worth my time to finish the project.

I also lacked the hardware to develop it further. I reached out to Marcel as it was his videos that inspired me to look at this. Obviously its very difficult to develop without the right setup, I'd change some things then wait on his answer if it worked etc... Basically impossible. The fact that the community didn't get behind the project means it will probably be stuck in early access forever.

Here is a video of it being used with the Valve Index stereo camera. You need to understand he could see this in his HMD it all runs in real time with no perceptible performance hit:

There are obvious problems with the chroma key needing improvement and the big one is that there is no stereo rectification so the video from the camera gives you a cross eyed effect.

All of this could be fixed but given the app cost me more to make than it earned I need to focus on my other software that pays the bills and all that mundane stuff.

The VR market is too small. The subset of the VR market willing to hang up a green screen and put up some studio lights just to see whats around them is even smaller. Sadly.
 

Latest News

Article information

Author
Julian Strasser
Article read time
2 min read
Views
24,826
Comments
78
Last update

How long have you been simracing

  • < 1 year

    Votes: 319 15.5%
  • < 2 years

    Votes: 219 10.6%
  • < 3 years

    Votes: 213 10.3%
  • < 4 years

    Votes: 160 7.8%
  • < 5 years

    Votes: 278 13.5%
  • < 10 years

    Votes: 239 11.6%
  • < 15 years

    Votes: 155 7.5%
  • < 20 years

    Votes: 120 5.8%
  • < 25 years

    Votes: 92 4.5%
  • Ok, I am a dinosaur

    Votes: 266 12.9%
Back
Top