Reader Submission #135 – The Use of Ballpark Figures

28724869973_b6587c4d8b_oOnly a few short weeks ago, the sim racing world was turned upside-down when Norweigan drifting personality and avid sim racer Fredrik Sorlie leaked a conversation between himself and Stefano Casillo of Kunos Simulazioni, in which the sim racer found himself on the receiving end of an aggressive virtual lashing from an otherwise respected developer within the sim racing community. While most of our readers rushed to take sides and either publicly blasted Stefano or accused Sorlie of being in over his head when it came to discussing tire behavior, lost in the community-wide argument was what the actual conversation centered around: tire behavior.

Casillo argued that the data and calculations powering the tire behavior in Assetto Corsa were the most important pieces of getting the virtual Ferrari on the screen to feel like a proper car driven to the edge of the tire, but Fredrik stated that his semi-random numbers inserted into the INI file – primarily the result of several trial and error experiments – produced a much more realistic range of vehicle dynamics on corner entry, which mirrored his own time spent blasting around the Nurburgring Nordschleife with his life on the line. Or, you know, something to that effect.

Today’s second Reader Submission comes from Richard Wilk, the in-house physics guru for rFactor’s Historic Sim Racing Organizationor HSO for short. The HSO website specializes primarily in full-length online races ahd championships held in machinery from an era of motorsports that placed speed over safety, either creating their own mods from the ground up, or re-building popular historic releases from the ground up to iron out their flaws. The website recently completed their highly competitive 1973 USAC championship to close out the 2016 calendar year, and are currently in the process of preparing for a 1980’s World Sports Car Championship event at Monza. Though these guys don’t receive much publicity on mainstream outlets, they’re busy as hell on their own little corner of the internet – consistently managing to acquire full grids for each and every event they hold.

ob_755803_cg1gkhAccording to Richard Wilks of HSO, you need more than just hard numbers – as Kunos Simulazioni have ruthlessly claimed when discussing tire behavior in private with real race car drivers – to create a convincing rendition of a virtual car, and it’s foolish to dismiss feedback from people who have driven the real thing, even if it goes against your own data. You’re building an experience, not a space shuttle.


1dac9a26393743cf75db3c55da1854146a8057d6Hello PRC! I’ve returned with another submission about the process of creating cars for all of your favorite simulators, but this time I’ve been a bit more outgoing than usual, and I’m finally comfortable revealing my name. You can read some of my past submissions HERE and HERE.

There was a lot of talk recently about Stefano Casillo from Kunos Simulazioni refusing to hear and even offending a guy with massive real life experience. To me, this is beyond unbelieveable. No, I don’t feel it’s appropriate to call names or question what Kunos are doing behind the scenes (though it may be a bit justified), but I’d like to explain to readers why this is all so preposterous to someone like me, who willingly spends his free time learning about cars, and creating a convincing set of physics for others to enjoy in a competitive setting.

As a physics modder, I can say that having a guy who not only drove the real thing, but understands how to be successful in a consumer simulator as well, and can flawlessly translate skills from one to the other, that’s pure gold. It’s already hard enough to find interviews or accounts from drivers detailing the real life experience because it’s not something auto racing journalists typically ask – they’re concerned about race strategies and other marketing things – so finding a guy willing to drive in a simulator for an excessive length of time  and even show you the way that the car behaves by modifying the files himself… I have to say I’m a bit jealous of Kunos that they have fans willing to go through that lengths to help the developers.

So for Stefano to shoot these people down… It’s very dumb. Honestly, incredibly dumb. But this gets much worse. You see, even if he believes he has his tire model numbers absolutely correct, he’s putting too much faith into two really dangerous categories:

  • That his physics engine properly translates those numbers into correct forces in all situations.
  • That his tire model is already perfect, or realistic.

Looking at point number one, I guess Stefano’s pride must have been hurt to lash out at Fredrik like that, so it’s no small wonder he doesn’t even question that his physics engine functions perfectly in all situations. But point number two is something he should very well question, because nobody, and I repeat nobody, can claim to have tires nailed in sim racing. And this is where feedback is most important.

When I sit down and work on a car for HSO, and this entails everything from helping with a scratch made mod our guys created down to every last lug nut on the wheel, all the way to tweaking an existing mod that people like but doesn’t drive very well, tires are the absolute last thing I mess with. You can do everything else right or get it somewhere in the correct ballpark, but tires? Its not just the grip. It’s the load sensitivity, the slip angles, or the relationship between front and rear slip angles, and how that all translates through the flawed or incomplete tire models we have, into car movements. This is a massive grey area, and you can’t rely solely on numbers, especially because those numbers powering other parts of the physics engine – or data that has to be extrapolated from other pieces of data – are not 100% reliable in the first place. This is where accurate feedback is crucial. Too many times I see things other modders have gotten wrong, because people just assume things about these cars, and never read or were bothered to ask people with legitimate experience.

I can understand modders getting this wrong, because Porsche or Ferrari haven’t given them free reign of their private garage, nor do they have the budget to acquire sensitive data or take these cars out to a track for firsthand experience, but developers themselves? A team who are supposed to know the inner workings of their software? It’s really inexcusable.

How can quality mod teams for Assetto Corsa exist, if the people creating vanilla content behave like this? They should be setting an example, not being yet another “I never sat in this car in my life, but I know better” autistic manchild.


1acfa9983bd05987f27314b3b2f1d1561e479838Even though we’ve sort of moved on from Stefano’s meltdown over Fredrik’s feedback and what it indicates about how Kunos Simulazioni operate, you raise an interesting concept that I’m sure the readers of PRC will appreciate (compared to a submission we posted earlier today, anyway).

When tires are still a bit of a black art that no single developer team – let alone real world car makers – have been able to master, why are Kunos behaving as if raw data and numbers they’ve set in stone are the answer to producing an authentic virtual recreation of performance driving? Consumer racing simulations – the ones we can buy off store shelves – are an approximation of vehicle dynamics using as much real world data that can be applied within the software, and then filling in the blanks with reasonable guesstimations. But physics engines themselves are an approximation of real life, using numbers to replicate the laws of the universe, so there’s no absolute guarantee the software powering these games is one hundred percent correct before we even place a car on the track.

Therefore, there’s no reason not to be open about feedback from avid sim racers with real world driving experience willingly plucking numbers into the game just to see what happens, because they might actually be onto something. And sure, let’s say after a ton of testing, their feedback results in experiments that are wholeheartedly inconclusive. That’s okay. It’s not a knock on you as a developer or as a person, it’s not them trying to undermine your years of obsessing over vehicle dynamics textbooks, it’s them saying “it doesn’t feel right to me, can we try going back to the drawing board so your software benefits me more on the real track than it already does?”

Unless there is something seriously wrong with your emotional state where even the slightest bit of feedback triggers immense hostility to anyone who crosses your path, this is how you improve the simulation value aspect of your simulator.

Advertisements

36 thoughts on “Reader Submission #135 – The Use of Ballpark Figures

  1. Stefano sounds like all these retarded shills that think numbers translate into the correct behaviour, because of course the simulation is perfect and takes everything into account. If the data is in and the lap time is correct, it must be a perfect simulation!

    Even if it drives like Shift.

    With a controller.

    Naked.

    Hopeless ‘tards.

    Like

  2. The thing is that Fredrik is an experienced driver for his own car, not for the rest of the cars in AC’s roster. If Kunos would be simulating Fredrik’s car for sure the devs would listen to every bit he has to say, but they aren’t doing his car. They are doing cars Fredrik hasn’t driven at all or enough, to say in detail what’s wrong. Kunos does listen to what amateur and professional drivers have to say (cmon, even Kunos staff drive cars on a race track), but data+technical simulation > driver feedback.

    I’ve never seen the devs react bad to other driver’s feedback, but stefano got irritated at fredrik’s pretend solutions.

    Like

    1. “The thing is that Fredrik is an experienced driver for his own car, not for the rest of the cars in AC’s roster”

      Oh fuck off ya numbty, how the fuck does a guy who has raced, ralleyed and drifted, driven the nords in several cars, and is a INSTRUCTOR for BMW,Porsche and AMG, adn also does stunt work including movies, morph inside your head as irrelevant because “his own car”? you muppet, hes a fucking dream to have info from for a dev like Kunos or any sim dev, who are these other pros Kunos are consulting?

      Liked by 2 people

      1. I’ve asked you this in another article, but it got buried.

        Given your troubled youth with people stabbing you and all that, were you part of the IRA by any chance?

        Like

    2. Even tho I’m actually fast IRL I’ve been slagged off because apparently my car isn’t expensive enough.

      Tyres are tyres, doesn’t matter what car they’re on. When literally every cars fall on their face due to the tyre dynamics then it’s pretty clear what’s going on.

      Like

      1. But tyres are different, not all tyres are the same. So, was Fredrik suggesting changes to the Street tyres of v7 TM or to the entire range of tyre options available in the sim?

        Like

  3. Stefano strikes me as the kind of person who thinks he’s always right about everything, and is therefore unwilling or unable to consider anything that contradicts his stance on a given issue. This is likely an attitude that belies a deep insecurity in his work. I’d almost feel bad for the guy if he weren’t such an ass.

    Like

    1. He’s humble enough to recognize the simulation was missing something when people bring him clear points, data, or some sort of evidence the physics devs can work with. Otherwise is just more of the usual feedback everyone can give.

      Like

  4. I’m surprised that Stefano would be arguing on the importance of data over feel, because I could dig up quotes where he says the exact opposite.

    Like

    1. It was practically the default response of Kunos and AC shills “fuck the data, its all about the feels man”, apparently changeable when suits.

      Like

  5. He’s going to reply to this in his little forum, breaking 505 games forum’s rules and making Trump sound like a reasonable man, to get his narcissistic circlejerk going as he tries to bash everyone who has criticism.

    And don’t be fooled. He says to ignore PRC and yet, he is always reading the blog.

    Like

  6. Great article. There’s a reason that race teams do actual testing. If everything could be accurately predicted with CFD and physics textbooks, there would be no need for testing at the track with your drivers.

    How many times did I read these words in Autocourse: “We still don’t understand last year’s car”. And this is coming from the people who *designed* the car.

    Like

    1. “There’s a reason that race teams do actual testing. If everything could be accurately predicted with CFD and physics textbooks, there would be no need for testing at the track with your drivers.”

      I like this analysis.

      Like

    2. Trying to be ignorant here or it wasn’t on purpose?

      Race teams setup a car therefore they need track testing, sim devs are creating a simulation software. The same way that in a sim you do track testing to setup your car for an online race, real race teams do track testing to setup their car.

      Like

  7. simracing is made of cocaine and steroids why do you think it would turn up in a constructive dialog. The Guy needs to read a Book and shut up 😀

    Like

  8. what everybody forgets is that frederiks initial post had that typical simracing tone with some hidden instults – something about realism not being the strength of kunos. I think that was the trigger for stefanos reaction. If you have something serious to say be serious and do not insult the devs

    Like

  9. I do not have the experience that Fredrik Sorlie and Richard Wilk have and from there profiles I am sure they are not wrong.
    But Geese! This site is just makes me laugh, always “Flogging a dead Horse” with a carrot stuck up the ass.
    When you have every Tom Dick and Harry who talks about things they obviously do not know sh1t about and demanding this and that form a dev, who has been open to the community (rather than like some devs behind closed doors) Then when the few who do know their stuff come along and put something out in the way he did, it just seems like another Tom Dick and Harry. No wonder Stefano blew it. Now I am not condoning his behavior. I work with several Italians and every now and again there are some moments, yeah from both sides by the way, but no one dedicates their life trying to bring down the other continuously. Flogging a Dead Horse every opportunity that arises.
    My guess that James was responsible for the Utter Crap that was posted about Assetto Corsa on Wikipedia some time ago. He also kept on deleting anyone else’s input until thankfully this was corrected with real facts and references have been put in place.
    Real facts and numbers count. Just looking at one source, Steam, “STEAMCHARTS An ongoing analysis of Steam’s player numbers” and Steam Reviews AC most of the time above any other racing/driving/sim in the charts 88% reviews are positive 83% recently. I am sure Stefanos behavior did not help recently. Reading the negative reviews it seems a lot has been influenced by this “alternative facts” site that just post the one sided take on any sim forum that has booted them from their membership. This blog with fake adverts is just hilarious!

    Like

    1. Typical failing to fook any further than one’s self belief. In the day some people believed the world was flat despite the facts. There might even be some who still do today!

      Like

        1. I had to look up this stuff to check if it was serious or a joke. Then I thought that it´s the same World where a buffoon is elected to run the most influential country so yep, it can be.

          Like

  10. Great article.. I wish our modding team had access to real life drivers more often. We have had some help on a few projects in the past.. And it was so cool when the Champion of that series worked with our sim racers to help build one of our mods. I had the pleasure of working with him on the physics feedback portion of it.. It was a great learning experience. Since then, even thou we don’t have a real life drivers on a steady for other projects over the years, we do still try and read up on what real life drivers say, plus try to find all real life data as we can..

    Nice to hear other modders in the community voicing there opinions.

    Like

  11. Just because Stefano had a beef with one guy, who was trying to help, who just happened to make a statement at the wrong time, doesn’t mean that KS refuses to take info from real drivers.
    It has been said before the blow up is not condoned. Ok it has happened now get over it.
    Continual barrage of misinformation from James and the like will certainly drive anyone to the breaking point.
    There is something seriously wrong with your emotional state if one devotes their time in creating a web blog to deface someone else who did not give you what you wanted a long time ago.
    You might learn a thing or two form people like Youtuber Empty Box who gives a balanced review or opinion. You just might get a couple of paid advertising opposed to the fake shit.

    Like

  12. This is based on an overly simplistic model of right and wrong, I think.

    First off, there are a lot more wrongs than rights. When you’re talking about car physics… there’s one right model, and a million wrong ones. When you tell someone they’re wrong, that doesn’t mean you know the right answer – but you can know for a fact that theirs is not close. If they claim a Porsche 911 makes 900 horsepower rather than the 481 modeled ingame, the real world figure being 487 does not make the complaint right.

    Second, complicating that, you can add two wrong things together and get closer to right than if you have one wrong and one right added together. I imagine this is what Wilks mainly focuses on – you get the real-world driver to tell you when the sum of the parts is correct, and it doesn’t matter which parts of the underlying model are correct or not. The key to this is, it only applies on things where you don’t know the right answer of either input – so for example if I say “x + y = 15, pick values for x and y” then guessing x=10 & y=5 is better than guessing x=8 & y=10, even if the actual values are x=8 & y=7.

    But in either case, you don’t have to know that your own solution is correct to identify specific things that are wrong. If someone sets up a tire so that it has constant grip at any temperature above 80C, it’s plainly not correct.

    If someone comes to you and suggests making a change to the tires so that their grip is constant above 80C, it’ll make the car handle more realistically, you have a couple fairly reasonable options.
    1) is to tell them they’re wrong, and look for a plausible change with the same effect
    2) is to check for bugs in the feature that could cause their unrealistic inputs to work out.
    There’s of course the third option which is not something any dev should do
    3) agree with them that their suggestion is an improvement, make the change

    Ultimately you should be willing to change known-correct values by small amounts to account for tolerances – sometimes a small difference makes a drastic behaviour change, for example even single percent changes in brake bias affect understeer quite a bit when it’s near balanced – but when someone throws something completely in the wrong ballpark, that’s not evidence to change anything at all.

    Like

Rap battles between users are encouraged; losers will be IP banned for 24 hours.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s