VR? No Buy! Get Me Mixed Reality!

MR_RD.jpg
"No VR, no buy" has become a staple of new sim racing release reviews. Even F1 22 now offers the, to some, elusive technology. But I'm really not about that. I'm in for the (hopefully) next hardware development. Mixed Reality.

Almost a year ago, Yannik Haustein asked the question: "What Do You Think Is The Next Big Hardware Development?"

And my answer is: "I don't know, but I wish it were Mixed Reality!"

So let's explore this mythical world.

Why We Need Mixed Reality​

Some of you might be asking: "But why?"

Well, there's 1 simple problem with VR for me: I cannot actually see what my hands are doing. I wanna see where my fingers are; which button I'm pressing on the wheel or button box (aka 2nd keyboard).

I want to see.

I want more immersion.

And I know it's possible. See it as a free business Idea. I just don't know if it's a good business idea.

How it Could Work​

Mixed Reality as a theoretical concept in sim racing has been around for a while now. I mean, just look at this video from 2016!


Now while some of you might know this channel already, if you don't, here's a quick run-down.

The channel owner, a German man by the name of Marcel Pfister, edits recordings of himself doing VR racing. All the while he sits surrounded by a green screen and later edits the VR surroundings into the video.

Now here's the deal: I want that. But REALLY working.

The theory is simple: Have a VR headset with cameras in front. The cameras just project the images they capture, excluding the green screen. That would be live projected with the interior of the car.

Sounds simple, right?

Why It's Not That Simple​

But it isn't. Every new development needs time and money. And the question would be, how big is the market for this?

And the, quite frankly, deflating truth of my thought process is:

Not very big, at least in the home-usage sector.

Commercially it could be used for driver training lessons or the like. So the potential users would be every vehicle driver.

But in the home sector, can you think of more examples of usage than, say, sim racing and Farming Simulator?

I can't. (But if you can, let us know in the comments, please.)

And a lack of possible monetary reimbursement automatically means a lack of investment. Sad, but what can you do?

What YOU Can Do​

Now let me be a bit selfish for a second. I know this is my wish and all, but if I have sparked your interest in mixed reality, just spread the word. If enough people wish for it, sooner or later, someone will invest R&D into this field.

And that could fulfil the wish (hopefully) not only I have. The (possibly) most immersive sim rig ever:
  • Motion Rig
  • Green Screen
  • Mixed Reality
But until then, would you be interested in a mixed reality environment like this? Let us know in the comments down below (and if you are, please share the word)!
About author
Julian Strasser
Motorsports and Maker-stuff enthusiast. Part time jack-of-all-trades. Owner of tracc.eu, a sim racing-related service provider and its racing community.

Comments

Assetto Corsa's CSP mod has actually recently added the ability to track hands and fingers, so you actually can see your own digits now with a predicted arm. We aren't at the stage where we see ourselves in the car with no gloves and sat there in jeans but its getting closer. But that is still Virtual Reality.
 
Premium
I def see some uses for mixed reality but sim racing, I wouldnt want to see my button box, I wouldnt want to spoil the view in VR for my open wheeler or obscure the dashboard of another car with something that would generally look out of place in a real race car.

I am certainly looking forward to things like hand tracking being more widespread so I can interact with my virtual car/plane in a very easy manner without hand controllers.

I am sure 9 times out of ten I would rather interact with a model of the real thing than see my less impressive button box.
 
D
Nice article. I wholly agree with you and would like to see development of mixed reality. Not only would immersion be improved, but I think I'd actually drive much better when I'm able to see my hands and steering wheel. I just hope it's implemented well without any visual lag.
 
I skimmed through the article and I still have no idea what it's talking about. What is "mixed reality"? What games support it? Can you provide some videos or explanations about it? Are you talking about when we, as players, get to race against real-life people while they're racing in real life? If that's what you mean then I think it's very gimmickey for the sole reason of real life having completely different physics than any game. It'd be no different than racing someone with one player playing iRacing and the other player playing Assetto Corsa - no point, meaningless. I think Simbin or Raceroom or someone tried this concept years ago. It had a name but I don't think it was called "mixed reality". It was a pointless gimmick.
 
I skimmed through the article and I still have no idea what it's talking about. What is "mixed reality"? What games support it? Can you provide some videos or explanations about it?
I mean you could read the article or... watch the video it links to rather than speculate and go off on incorrect assumptions.

"Mixed reality" means virtual reality but projecting some aspects of the real environment (in this case the driver's hands and wheel) into it. "Augmented reality" is the other way round, projecting virtual elements into your real environment.
 
Premium
Interesting article. I thought you meant something like what they do with RC drones and VR... but with RC cars and a simrig? Something like this but in VR for the immersion:


I remember seing Linus Tech fiddle together a kit for this purpose, which worked to some degree. Nintendo had a go at XR (AR) with "Home Circuit" for Mario Kart on the Switch (true fun stuff for a few rounds) but hardware was a thing, and since their VR is primitive (and literally "hands on") it wasn't optimal XR... still, some people have had an attempt at improving it. This concept, with RC cars on a real track and with a proper wheel could be serious XR fun:

 
Last edited:
Premium
Interesting article. I thought you meant something like what they do with RC drones and VR... but with RC cars and a simrig? Something like this but in VR for the immersion:
That's interesting too. Honestly that is something I wasn't even aware of. But it makes sense.

I still remember the trial they did on Red Bull Ring (last year?) where a person from a nearby engineering plant steered an actual car around the track from away.
 
Premium
Well... Hori's "Mario" (pro) wheel works with Home Circuit so, technically, it can be done at the low-spec range... :D

 
This business about wanting to see your own hands.......Go for a drive in a real car. Start driving quickly. Are you looking at your hands or the wheel? I hope not.

I'm happy with the completely enclosed vr world. I'm like a blind man and have learned where everything is. Once again though, do you look at your real gearstick when driving, do you look at your indicators? No, you just use them.
 
First of all we need more optimization for true VR. Even with the maximum hardware power you can buy at the moment I have to run ACC @60hz on my G2 with acceptable visuals. More engine performance is so much more important than seeing my M4 steering wheel and my hands while sitting in a 720s McLaren. XD
The Sim Devs should focus on VR only and don't create new problems.

I love VR because it gives me the possibilty to leave the real world, to be somewhere else and not in my living room. I don't need and want Mixed Reality.
 
about. What is "mixed reality"? What games support it? Can you provide some videos or explanations about it? Are you talking about when we, as players, get to race against real-life people while they're racing in real life? If that's what you mean then I think it's very gimmickey for the sole reason of real life having compl

Mixed reality means merging of real and virtual words so that physical and digital objects can exist in the same visual space.

Varjo XR-3 can already do this but it is not a consumer product:


Here's another fun little lab experiment they posted a while ago:


Not perfect yet but it is certainly getting there. While the target audience is mainly large enterprises such as Lockheed Martin, Boeing, Volvo et cetera they apparently partnering with another Finnish company – and one which you may have heard of – Simucube.

So fingers crossed maybe we'll see something interesting on that front within the next 10 years. Varjo Aero is already the best VR headset for sim racing, so hopefully they will be pioneering consumer grade mixed reality as well.
 
Premium
I'm with metainwood, no way do i need anything more in my VR headset than what i already have. Hands on wheel, got it. Gear change, got it. It still amazes me when people say VR is not ready yet so they continue to drive through a window. Still the best kept secret in sim racing despite telling the whole world. Been said many a time, could not go back to screens .
 
Mixed reality makes no sense for sim-racing. Why would you want to see your steering wheel when one of the main goals of VR is let you see the virtual cockpit of any car, including its steering wheel. Oh, yes, I know: you spent a lot of money in a steering wheel full of buttons, labels, rgb leds, which looks like the real thing and all that is lost when in VR. If that is your concern, don't use VR and you will be happy looking at your beautiful cockpit, your button boxes, dash screens, etc. The only point in which I agree that the integration with the real world is interesting is the ability to see how you move your hands, but VR is also able to handle that and I'm sure the new headsets will be able to identify your hands and fingers and simulate them in the game without the need of anything else.
 
I mean you could read the article or... watch the video it links to rather than speculate and go off on incorrect assumptions.
I did. When I clicked the video, all I got was a trailer for F1 2022. It's not the video I see now.

Mixed reality makes no sense for sim-racing. Why would you want to see your steering wheel when one of the main goals of VR is let you see the virtual cockpit of any car, including its steering wheel. Oh, yes, I know: you spent a lot of money in a steering wheel full of buttons, labels, rgb leds, which looks like the real thing and all that is lost when in VR. If that is your concern, don't use VR and you will be happy looking at your beautiful cockpit, your button boxes, dash screens, etc. The only point in which I agree that the integration with the real world is interesting is the ability to see how you move your hands, but VR is also able to handle that and I'm sure the new headsets will be able to identify your hands and fingers and simulate them in the game without the need of anything else.
Why would I want to see some, fake, pre-programmed, scripted, cartoon hands instead of my own? Same with my steering wheel, shifters, my body, arms, buttons / switches, etc.

It's not about looking at your cockpit. Trust me, I'm in the same boat as you, I laugh when I see people with 200 different gauges, tablets, cell phones, etc. so they can have 30 different rev-lights, see a list of laptimes from 35 different drivers simultaneously, and 500 other pieces of info as if they're a scientist about to conduct an experiment. On the other hand, lots of us don't want our physical bodies to be replaced my some cartoon videogame graphics. There's also the fact that you need to see buttons and switches on your wheel and button box. The wheel you can get used to but you need your eyes a lot of times for the button box (even more so if wearing gloves).

I'm a gigantic fan of 3D / stereo vision gaming but one of the main things holding me back to going to a VR headset and therefore still using Nvidia 3D Vision on triple screens is that my cockpit disappears with a VR headset.

I don't play racing videogames because I want to be in a videogame world. I play because I can't race in real life but I want to feel like I'm doing it in real life as much as possible rather than feel like I'm inside a video game world doing it.

In other words, I don't want to be in the game, I want the game to come to reality.

Once graphics become more life-like and VR headsets can track your body better (fingers, arms, joints, etc,) and with better detail & fidelity, and therefore the driver is doing what you truly do rather than relying on fake animations, then it may be a different story.
 
Last edited:
isn't mixed reality more for the viewer watching rather than more for the user playing, hence the big green screen around everything in the video for drifting.
Do you mean AR to incorporate hands and say button boxes in your own setup? I watch things like beat saber in mixed reality and its cool but the user don't see all that just the viewer, as a broadcast thing mixed reality could be ok, imagine cutting to the in view driver in a broadcast and seeing them in car on track, but again that would only be for the viewer to see not the user
 
3 monitors, the poor man's Mixed Reality... :D
rich mans i would say.
3x 32" monitors + stand cost way more than an Oculus Quest2 or a HP Revrb

on the other side.
Quest has very good hand tracking, for steam devices there were some tags for the hands in development.
So its on the game devs to support this.
No need for expensive mixed reality headsets
 
Last edited:

Latest News

Article information

Author
Julian Strasser
Article read time
2 min read
Views
24,813
Comments
78
Last update

How long have you been simracing

  • < 1 year

    Votes: 94 12.6%
  • < 2 years

    Votes: 69 9.3%
  • < 3 years

    Votes: 74 9.9%
  • < 4 years

    Votes: 45 6.0%
  • < 5 years

    Votes: 104 14.0%
  • < 10 years

    Votes: 105 14.1%
  • < 15 years

    Votes: 62 8.3%
  • < 20 years

    Votes: 41 5.5%
  • < 25 years

    Votes: 37 5.0%
  • Ok, I am a dinosaur

    Votes: 113 15.2%
Back
Top