Another Attack of the Killer Floor Mats: Sarasota Edition

Dear Toyota:

Why did you buy back Tim Scott’s 2007 Lexus RX? We mean, really? You gave him a bunch of different reasons, but he doesn’t believe you. (We’re finding it a little hard to swallow, too.)

Awaiting your reply,

SRS

Here’s Tim Scott’s story. In early December, as NHTSA and NASA were putting the finishing touches on their reports saying that there is nothing wrong with Toyota’s electronics or software, Scott experienced an unintended acceleration event in his 2007 Lexus RX350, on his way home from the gym. Here’s the narrative that Scott, 46, the chief financial officer for the International Union of Police Associations, wrote: Continue reading

We Read the Report. Did Ray?

Last week, NHTSA pitched its two technical tomes on Toyota unintended acceleration at a pack of reporters, declared that the automaker’s electronics were fine, and ran away. Our esteemed Secretary of Transportation Ray LaHood then made the media rounds, grousing that the critics hadn’t read the report, which leads us to ask: Did Ray?

We’ve been reading it and re-reading it, and conferring with a wide range of technical experts – some of whom have extensive experience in engine management control design, validation and testing. And we gotta tell you, Ray, we aren’t ready to buy our kid a new Toyota.

Far from exonerating Toyota electronics, the reports by NHTSA and the NASA Engineering and Safety Center (NESC) confirm the paucity of the automaker’s safety diagnostics. The NESC team also identifies how the two signals in the accelerator pedal position sensor can be shorted in the real world – leading to an open throttle (aka, tin whiskers). Hell, NESC found the potential in three pedals – that’s a pretty significant percentage in a very small sample. Tin whiskers are such a serious issue that NASA has devoted considerable resources to studying them. They have wreaked electronic havoc on everything from medical devices to weapons systems and satellites. Yet, the NESC report treated the discovery of tin whiskers in a third of their pedal sample like a dead end, instead of a promising avenue of study. Continue reading

NHTSA Shuts the Door on Toyota Electronics in High Speed SUA – NASA, Not so Much

In his characteristically colorful way, Secretary of Transportation Ray LaHood told reporters today: “We enlisted the best and brightest engineers to study Toyota’s electronics system, and the verdict is in. There is no electronic-based cause for unintended, high-speed acceleration in Toyotas.”

LaHood issued this scientific proclamation based on the National Aeronautics and Space Administration “rigorous” examination of nine Toyotas in which the drivers complained about Sudden Unintended Acceleration.

Well, case closed.

We don’t think the story’s over – not by a long shot. First, we might argue that engaging engineers whose expertise is not automotive engine controls does not constitute “the best and brightest” minds in this particular instance. Second, no scientist worth his or her boots would make such a claim. In fact, NASA didn’t. That agency said:

“Due to system complexity which will be described and the many possible electronic software and hardware systems interactions it is not realistic to prove that the ETCSi cannot cause UAs. Today’s vehicles are sufficiently complex that no reasonable amount of analysis or testing can prove electronics and software have no errors. Therefore, absence of proof that the ETCSi caused a UA does not vindicate the system.”

We will be providing more analysis of the NHTSA-NASA report after we’ve had a chance to digest its findings. Stay tuned.

What We Know About Toyota Electronics

While NHTSA and NASA have been busy in their test labs, we’ve been busy doing some testing of our own. And, although our findings are preliminary, we’re uncovering important clues to the gaps in Toyota’s electronic safety net. We haven’t seen NHTSA’s report, but we’re hearing the sound of hands dusting themselves off and feet walking away. What’s troubling is examinations of the complaint data consistently show statistically significant increases in SUA complaints in Toyota models when equipped with its Electronic Throttle Control system. (See Quality Control Systems Corporation’s What NHTSA’s Data Can Tell Us about Unintended Acceleration and Electronic Throttle Control Systems for more information.) Toyota has replicated these incidents, as reported in field technical reports. (See Toyota Replicated Incidents for more information.)

Detailed evaluations of various Toyota models with ETC have revealed some fascinating design issues that demonstrate weaknesses in Toyota’s electronic architecture.  In short, their designs don’t have enough computing power to integrate ETC into the engine control and incorporate safety features needed to prevent unwanted events.

Because the algorithms that make up the software in Toyota’s engine controllers are overly simplistic they are incapable of providing a robust electronic safety net that is needed in these types of systems.

The simplified software strategies used in Toyotas demonstrate this lack of computing power and software.  It is important to note that none of the physical components need to change to accommodate safety features, they are software driven, yet these key software features are not there.  Following is an overview of some of the areas that we’ve examined related to design features that exemplify this problem.

Throttle learn/ Spring test:

There is a parameter in the Toyota engine controller referred to as the “Throttle Learned Value.”  Throttle learning is done to account for part-to-part variations and adjusts the sensor reading up or down by an offset based on the physical components.  The effect is that the entire pedal-to-throttle curve will shift up or down in response to this value.  This is very evident on pedal-follower type systems such as on the 2005 Camry.  What happens if the reference position is not what it was assumed to be?  In short, the entire relationship of sensor voltage to throttle angle can be skewed causing more engine output than the driver requested.  This condition has been documented to cause short duration UA events.

The spring self test is an important safety check performed by the engine controller when the key is on and before the car starts.  The controller opens the throttle very briefly and monitors the return spring closing.  This basic safety test is designed to check the response of the throttle body return spring to ensure throttle open / close response is functioning appropriately from a mechanical standpoint and gauges the dynamic response of the actuator before the engine is started.  There is no such test in the Toyota models.  This is important because it illuminates the lack of system integration between the electronic controller and the mechanical components they control

Pedal-Follower

Many Toyota vehicles with ETC use a pedal-follower system.  This system has significant limitations and doesn’t address the core concern which is engine torque output.  Nor do the subsystems in a pedal-follower system communicate in a manner that facilitates a more complete control of the engine.

Level Two Engine Diagnostics

Primary diagnostics are designed to catch faults conditions usually of an electrical nature.  Secondary, or Level-2 diagnostics, are rationality tests designed to catch unanticipated anomalies and to the monitor the basic functions of engine control system.  This is used to check the actual engine torque versus driver input, and mitigate if necessary.

Some Toyota ETC systems examined appear to have a complete absence of secondary or Level-2 engine torque diagnostics.  The lack of this safety net presents opportunities for undetected faults to create unwanted events.

Brake Override

Electronic brake override in an ETC system was originally a feature designed to address a physical stuck pedal condition.  If the pedal is stuck but sending a valid signal to the engine controller and the driver is braking, the accelerator pedal voltage signal is overridden (via an algorithm in the software) and the engine is returned to idle.  Toyota has “re-flashed” some late model vehicles with a brake override in recall 90L.  However, the company claims that a similar software re-flash cannot be accomplished on most of their vehicles because of the lack of computing power.

Cruise Control

Inputs on Toyota models appear to have no redundancy.  This is an atypical design compared to other OEMs.  This lack of redundancy creates the potential for failures and misinterpretations of the cruise control status due to potential electrical faults.

Accelerator Pedal Position Sensors (APPS)

Most manufacturers use a triple redundancy on the APPS.  The Toyota APPS design differs in the effectiveness of the second sensor.  While it is a separate sensor, it doesn’t have a different characteristic in the voltage slope which can result in “blindness” to certain external voltage influences.  There are also concerns associated with the calibration of the sensor agreement diagnostic logic.

(See Toyota Electronic Throttle Control Investigation: Preliminary Report )

Idle control:

Toyota allows a wide-range of idle deviation without setting Diagnostic Trouble Codes (DTCs).  Idle speed variations in excess of 2,000 RPMs have been documented in real-world conditions without driver input.  This level of idle generates a substantial amount of engine torque and can result in UA events.

If we take a moment to consider the big picture, we see regulators that have failed to regulate and the investigators have failed to investigate. They’ve set themselves back – and in the process – all of the motoring public.

Toyotas — Erratic By Design and the Case of the Duplicated Condition

Safety Research & Strategies’ continuing investigation into Toyota SUA is building momentum as new defect issues emerge from murky, unregulated vehicle electronics.

We have also found a document that suggests that Toyota lied to the driver in one of this issue’s most hotly debated incidents.

We’ll start with the document. You may recall Rhonda and Eddie Smith, the Tennessee couple whose testimony before the U.S. House of Representatives’ Commerce Committee last February transfixed the room. Rhonda, a retired social worker, recounted her harrowing ride in a run away Lexus ES350 over the hive-like buzz of clicking photographers. Wiping away the tears, Smith talked about trying every strategy she could to wrest control of her vehicle as it raced to 100 mph. Rhonda managed the Lexus to the side of the road, once the vehicle speed dropped to 33 mph.

[flashvideo file=video/rstestimony.flv image=”video/rstestimonypreview.jpg” /]

Rhonda had noted two clues to the electronic nature of the misbehavior – the cruise control light came on as the vehicle began accelerating, and, while at rest, the Lexus tried to start itself with no occupant inside. The wrecker driver later gave a notarized witness statement attesting to the vehicle’s attempt to turn over the engine by itself. Click on the thumbnail below to see the wrecker driver’s statement.

But, in April 2007, when NHTSA’s Office of Defects Investigation traveled to Tennessee for an inspection of her Lexus, ODI focused on the state of her brakes and her floor mats. After the Smiths shared their story publicly, auto bloggers, like Edward Neidermeyer of the dubiously-named The Truth About Cars, cast more doubt:

“One of the most important lessons to come out of the last two days of congressional hearings on the Toyota recalls is that blaming individuals for unintended acceleration is too tough a task for our elected representatives. And yet the more we learn, the more necessary it seems to take human error into account when dealing with unintended acceleration. Nothing illustrates this quite like the case of the very first witness to give testimony before congress. Rhonda Smith of Sevierville, Tn told the House Energy Committee, under oath, that her Lexus ES350 became “possessed” and that its brakes and transmission failed to respond at precisely the moment that the car accelerated out of control. “Shame on you, Toyota, for being so greedy,” she said, wiping tears from her eyes. But it turns out that the shame belongs almost entirely with Ms Smith.”

Actually, the shame’s back on Toyota.  Here are some facts (and documents to back them up).

Smith’s incident occurred on Oct. 12, 2006, and shortly thereafter, Toyota brought in a Field Technical Specialist to inspect the vehicle. According to Toyota’s internal documents:

*** NOTES 11/15/2006 07:46:06 AM PPatel Dlr notes” FTS INSPECTED VEHICLE AND WAS ABLE TO DUPLICATE CONDITION. FTS SPOKE TO CUSTOMER, AND CUSTOMER DOES NOT WANT VEHICLE BACK, CONCERNED ABOUT SAFETY. FTS FORWARDED INFORMATION TO LEGAL DEPT. AND CUSTOMER IS WAITING TO HEAR FROM LEGAL DEPT. CUSTOMER IS CURRENTLY DRIVING DEALER LOANER CAR

Click on the thumbnail below to see the case report containing the above notes.

We sent Toyota’s internal notes to Rhonda, for her records.

“My first reaction was elation. I feel like this vindicates what I said, because so many people had not believed me,” she said. “I know Toyota lied to me, but now I have it in black and white, and it may help other people who are trying to get their issue resolved.”

The Smiths say that the customer service representative, Ms. Patel, had conveyed this information to them in a phone call. They were so stunned they asked her to repeat it. But they never mentioned it to Congress, because on December 4, 2006, the automaker sent the Smiths a letter that said something different:

“You have requested the findings of our investigation. Your vehicle was inspected by one of our Field Technical Specialists (“FTS”) on October 24, 2006. Our FTS was unable to duplicate the unintended acceleration that you reported.”

Click on the thumbnail below to see the full letter.

Three months later, the Smiths lost an arbitration to have Toyota buyback the ES350, based, in part, on Toyota’s claim that the vehicle had not malfunctioned:

“The Manufacturer’s position was that the Manufacturer Response Form stated that the vehicle was inspected by Field Technical Specialist Leonard St. Arnand, and found to be operating as designed.”

Click on the thumbnail below to see the full arbitration decision.

On April 11, 2007, ODI’s Scott Yon travelled to Tennessee to inspect the Smiths’ Lexus. He noted at that time that Rhonda’s vehicle had an unsecured all-weather floor mat on top of her carpeted mat, which was how the dealership sold the vehicle to them, the Smiths said. ODI did a test drive and an examination of the brakes, but did not perform any electronic evaluations or testing. Rhonda’s incident was folded into Preliminary Evaluation 07-016 and later Engineering Analysis 07-010 as a floor mat-induced event. Under hectoring from Congress, NHTSA eventually bought the Smiths’ Lexus for an exorbitant amount of money – years after the event, a change in ownership and numerous service interventions. NHTSA has not released any information about the Smith Lexus since.

Click on the thumbnail below to see NHTSA’s full Vehicle and Incident Site Inspection report.

To recap:

  • The driver and other witnesses describe a vehicle with multiple electronic malfunctions.
  • A field technical specialist duplicates her event
  • Toyota tells the customer the opposite and fights to skirt any responsibility.
  • NHTSA uses the event to bolster its floor mat interference theory with scant evidence.
  • Public ridicules driver that tries to raise awareness about a safety hazard.

What’s wrong with this picture? The agency’s inference that floor mats were to blame is now belied by testimony of the driver and Toyota’s internal documentation, which acknowledges duplicating her event, long before ODI got involved. Never once does Toyota ascribe her incident to floor mat interference. Surely, if Toyota could have blamed the floor mats, it would have.

On to the subject of Toyota’s designs.

We must remind you that our position has always been: This is a complex, multi-root cause problem. There are the runaway at high speed complaints and inconsistent throttle output complaints; i.e., my vehicle suddenly lurched forward or backward.  Some of these incidents can clearly be attributed to floor mat interference and driver error; however, many complaints don’t drop neatly into either category, based on witness accounts and other physical evidence.

Examinations of the complaint data consistently show statistically significant increases in SUA complaints in Toyota models when they were fitted with its Electronic Throttle Control system (See Toyota Sudden Unintended Acceleration: We’ve Got the Numbers).  And the nature of complaints that fall outside of mechanical interference and driver error strongly suggest electronic control system issues are at the root of many unintended acceleration events.

Detailed evaluations of various Toyota models with ETC have revealed some fascinating – and troubling – design issues that demonstrate huge weaknesses in the Toyota fault detection and fail-safes that are intended to catch and prevent unintended consequences.

For example, several intermittent and progressive mechanical throttle body problems lead to the engine racing without driver input and unpredictable accelerator pedal response.  These conditions are often not detected by the vehicle Engine Control Module (ECM) and no error codes are set.  The outcome is sudden high engine RPMs without driver input to the accelerator pedal or a pedal that responds with much greater acceleration than what the driver input requested (i.e, a driver depresses the pedal approximately 10 percent and the response is 35 percent causing the vehicle to lurch forward unexpectedly).

These conditions illuminate the holes in Toyota’s primary diagnostics and, more concerning, the lack of engine torque safety features (used by most other manufacturers) to catch problems that fall through the first safety net.

This adds to the already troubling known failure of the ECM to detect loss of accelerator pedal sensor redundancy (See Toyota Electronic Throttle Control Investigation: Preliminary Report )

In the larger context, what appears to be occurring is the confluence of issues that creates a vehicle that is more prone to inconsistent and unexpected behaviors.  These findings point to a warmed-over engine control system that doesn’t manage torque very well with ETC merely tacked on.

Toyota SUA has evolved into a high-stakes problem. And it’s not just about the money lost in auto sales or won in litigation settlements. The real importance in cracking this nut is learning what makes a robust design that relies on electronics instead of mechanics to perform safety critical functions.  The implications of this will set the stage for future regulations and defect examinations for years, and NHTSA’s relevance in this arena depends on it.

So What About the Defects?

In 2010, NHTSA levied nearly $50 million in fines against Toyota for flouting the recall regulations in three separate instances. The total represents the largest single fines in the agency’s history – and, (although we haven’t checked) quite possibly more than the agency has ever collected from any and all automakers in 40 years of existence.

This tough stance on recall timeliness is welcome – but does not resolve the larger issues raised by Toyota unintended acceleration – namely how defects are defined in the era of automotive electronics and how such defects are investigated when they are rare, multi-root-cause, and potentially deadly?

The dribble of documents released by the Multi-District Litigation and Congress so far show that UA has been duplicated by Toyota technicians and, contrary to attempts by Toyota advocates and agency investigators to pass off all incidents as driver error, sticky pedals, big shoes and floor mats, there are instances when reliable technical personnel take the vehicle for a test spin and experience UA with no pedal involvement. In fact, we have discovered that Toyota techs were able to duplicate UA in one of very public and widely debated case – but lied to the consumer about it. (We’ll feature that story in a future post.) Continue reading

Double Ding for Toyota

Toyota closes out 2010 by shelling out another $32.4 million to the government for tardiness. The two fines – for failing to recall its floor mats and defective relay rods within five days of determining a defect – were disclosed yesterday.

Three record fines in one year ain’t beanbag. In all three cases – the relay rods, the accelerator pedal and the floor mats – Toyota had recalled the affected vehicles overseas months before it got around to recalling those components here. It’s refreshing to see the agency enforce the law. But penalizing a manufacturer for failing to file a timely defect report only requires counting to five. The agency will greet 2011 with the much more complicated issue of unintended acceleration hanging in the balance. We’ll address that in a future post.

In the meantime, back to the fines. The details were MIA. NHTSA did not say when it thought Toyota had a duty to recall those components. Toyota didn’t admit it did anything wrong. Since the agency hasn’t made its case for the penalty to the public, the Safety Record Blog will do it for them. Continue reading

The Corrections

Apparently, when Toyota isn’t conducting secret polls to destroy our reputation, it’s reading our blogs. (We blush.) Yesterday, we got an e-mail from Mr. Mike Michels himself! Michels, Toyota’s Vice President of Communications, asked us to correct a paragraph in our post entitled “Toyota’s Quiet Buybacks Speak Up.”

We quoted an allegation from the Multi-District Litigation, which purported to show that Mike Robinson, Toyota’s Technical Supervisor of the Quality Assurance Powertrain Group, Toyota/Lexus Product Quality & Service Support, was an Avalon owner who had experienced an SUA incident. This is what we reported: Continue reading

Is This the Best You Can Do?

Time to gas up the Toyota PR engine. Yesterday, ABC News broke the latest allegations of Multi-District Litigation – that Toyota technicians had duplicated owners’ SUA events in incidents that didn’t set DTCs, and in two cases, bought back the vehicles and swore the customer to secrecy. (Remember that expensive advice Toyota bought in February from the BSG Group? “Portray transparency, open and honest.”) In response, the World’s Number One Automaker sputtered: Continue reading

Toyota’s Quiet Buybacks Speak Up

ABC News got a hold of the amended complaint in the Multi-District Litigation and is reporting that Toyota bought back two of its vehicles after its own technicians replicated the SUA events, which were not caused by floor mats, driver error or sticky pedals. According to the ABC story, Toyota bought a 2009 Corolla in Texas and a 2009 Tacoma in California, urging the owners to keep quiet about it. Continue reading