I am lucky enough to be in a position where I don't really have any other hobbies that cost me much (if any) money, so I have the good fortune to have been able to sink a fair amount of disposable income into my setup. My rig is far from "elite" in that I don't really have anything in the way of "boutique" type upgrades, but I'm guessing in the grand scheme of things my setup is a bit above average.
First and foremost, I will say in my experience I would not go chasing these types of upgrades to get big chunks of laptime improvements. As with real racing, far and away the biggest factor is the driver. If you've got full seconds to make up, as much as it sucks to hear it: it's you. (And, if you are unfortunate enough to be chasing down an alien or aliens, you are probably never going to make up the gap...unless you are yourself an alien!) Once you are to the point where you are trying to make up tenths (not full seconds), it's probably time to start fiddling with car setup options (I made the mistake of trying to tackle this hill way, way before my skills were ready for it).
Knowing what I know now, if I had the hankering to put some money into my sim racing hobby, the first money I would have sunk into it was buying a copy of the Skip Barber book "Going Faster!" You can also find an abbreviated version of much of what is covered in the book in a free-to-watch YouTube video by the same title. The book has far more depth though and, for me, reading and studying it was the number 1 thing that "flipped the light on" for me (I'm still not "fast" by any stretch, but I at least know the difference now between what makes a slow driver slow and fast driver fast and can do enough of the "fast" things to get by and enhance my enjoyment of the hobby).
Now, on to my hardware upgrades. One of the interesting things I'll mention is that I am in what I assume to be a somewhat unique position in that I've handed down all my cast off gear to my son. So, right next to my rig in the basement, he has his own rig with all of the components I have moved on from - that means periodically I have jumped over to his rig either right before or right after lapping on my to see how it compares.
I don't regret any one upgrade I've put money into - each upgrade I think has contributed to my personal immersion and/or laptime performance (much more the former than the latter, though). So, if I had to do it all over again, I am pretty sure I'd have bought all the same stuff...probably just in different order than what I did.
Wheel:
Old X360 wheel w/adapter (no FFB on PC) -> G27 -> Thrustmaster TX
If you are playing with a wheel with no FFB (as I was in the early days), I am quite confident you'll get a decent performance boost upgrading to a wheel with FFB support. Many/most sims at the minimum give very strong understeer warnings via the FFB effects. AC in particular does a very good job of giving you a very good sense of overall car balance via the FFB effects (even though IRL you aren't necessary going to feel this via the steering rack). Going from gear driven (like my G27) to a direct drive (like my TX) I think is not likely to net you much of a performance gain. The higher fidelity the FFB effects coming from the sim, the bigger difference I think you'll notice. rFactor2 for example feels very different (in a good way) on my TX than it does on the G27. The TX is much quieter and smoother, which helps with immersion I think. Performance gains in terms of laptimes *might* be there, as it feels like you get less lag between the effect coming out of the sim and into your fingertips, but I certainly didn't see my laptimes substantially increase.
Monitors:
Old 23" monitor -> New 27" monitor -> 27" x 3 (triples)
If you are like me, the first time you get the "FOV calculator" religion, you are likely to start dropping cash into monitors shortly thereafter. Going from 23" to 27" was, for me, an improvement that only satisfied me for a relatively short time. I went to a full on 27" x 3 setup pretty quickly thereafter and I am very happy I did. I do think there are laptime performance gains associated with upgrade, especially at particular tracks with very tight turns - for example, I really can't even drive Silverstone on a single monitor anymore because it makes "The Loop" essentially a blind corner for me. On the other hand, with triples, I can pick up the apex on my left-most monitor very easily and time my turn-in accordingly. Within a real race though is where the triple advantage really kicks in, I think - being able to see what's happening to either side of you is a big performance advantage (and, for me, a sizable immersion boost to not have to rely as much on virtual mirrors, opponent locator arrows, etc)
Shifter:
No shifter -> G27 shifter -> TH8A shifter
This for me is pure immersion. I am kind of a stickler for driving the car as closely as I can to real life from a transmission behavior perspective. Performance-wise, in my personal experience, more often than not prepare to be punished for insisting on (realistically) using a h-pattern shifter against opponents who in many cases will be using paddles (and often no clutch). Whatever - it's more fun to me to feel like I'm really experiencing what it's like to drive the car than to have a few tenths of performance (which isn't going to be enough for me to be even close to wining the race anyways!) The TH8A's price tag was a little hard to swallow, especially when I got the G27 shifter as part of the G27 "package" and I don't really think the TH8A is that much better. The main advantage the TH8A has is that it lets you simulate lever-style sequential shifting for cars where that is appropriate.
Pedals:
Crappy X360 wheel pedals -> G27 pedals -> T3PA Pro + Ricmotech load cell mod
Of all the crap I've sunk money on, pedals is where I think I could make the most legitimate performance gain argument. Clutch and accelerator are complete performance non-factors in my experience, but I think brake pedal behavior is critically important. Between the G27 pedals and the T3PA Pro sans load cell mod, I really can't say there was much of a difference. You can reconfigure the T3PA's between "GT" and "Formula" mode, but even a self-described "immersion junkie" like myself more times than not can't bring himself to futz around doing the reconfiguration work (I leave mine almost exclusively in the "Formula" mode just because I'm used to it). The "holy crap" moment I had though was when I added the Ricmotech load cell mod. Expensive little thing, but it really put the sim racing experience on its ear for me. My trail braking technique is SO much better now (definite laptime improvements) and also heel/toe is far more natural/consistent/effective (probably laptime improvements here and there - I blow fewer shifts, I don't over-blip as often). Immersion-wise, the load cell mod also makes the brake pedal feel much, much more like an actual brake pedal.
Cockpit:
Wheel clamped to desk -> Wheel Stand Pro -> Obutto Ozone
As far as the Ozone goes, it's kind of nice that it ties everything together - triple monitors, wheel, pedals, shifter, and seat all fit together in one nice, semi-adjustable package. Having a racing seat is a nice immersion boost for me, but I can't imagine it did anything for my laptimes (though in longer races where comfort becomes a factor, I could imagine being less likely to make a mistake if my back/butt aren't hurting).
Immersion also helps with the fun level. To closer feel that you are in a car instead of looking at one monitor with your living room or boy/girl room on the sides. 
