Automotive Software Application Lifecycle Management for ACES (ACES ALM)

Software eats the car: The four letters “ACES” stand for the major trends in the automotive industry: Vehicles are increasingly Autonomous, Connected, Electric, and Shared. Professional, automotive-grade software engineering is the basis for the software-defined car. At NTT DATA, we provide a comprehensive analysis of the business capabilities required to develop and maintain automotive software over the lifecycle.

Why did software develop such an appetite for our mobile vehicles - and why is it worthwhile for OEMs to serve a solid meal rather than fast food? Leading-edge application lifecycle management is essential in the automotive industry today. Therefore, in this article series, I will define ACES ALM before detailing the challenges in this area and introducing the ACES ALM business capability model in subsequent posts. If you are involved in shaping processes, methods, tools or organization (PMTO) for automotive software, this is for you. If you are in a hurry and don’t want to wait for the article series, please contact me directly.

What is ACES ALM?

Software for autonomous, connected, electric, or shared functionalities is not your typical enterprise application. An accounting system or common office software has no brakes that can fail. On the highway, it is difficult to simply open the task manager to shut down an annoying process. In-Car software is safety-critical, real-time embedded software running on electronic control units (ECU) in the vehicle. The following picture provides an overview of ACES software:

ACES is short for Autonomous / Automated, Connected, Electric, and Shared. The acronym ACES describes four major strategic pillars of automotive companies, especially OEMs. All four elements are part of the overall digitalization trend and require major innovations in IT hardware and software.

ALM is short for Application Lifecycle Management. As a process, ALM is about managing software from its initial conception, through the development and testing phase and ongoing support to its end of life. ALM tools provide support for activities such as software planning, requirements management, architecture design, software development, build, integration, test, deployment, operations and maintenance.

Combine the two and you get ACES ALM: ALM for ACES software.

And why is software eating my car?

More than 10 years ago, entrepreneur and investor Marc Andreessen famously published an essay in  Wall Street Journal on “Why software is eating the world”. The automotive industry has come to realize the truth behind statements such as “Software is also eating much of the value chain of industries that are widely viewed as primarily existing in the physical world.”

As a case in point, Volkswagen Group in 2019 has established CARIAD to increase the workshare of VW in ACES software from currently 10 - 15 % to 60 % – with currently more than 5.000 people working on it. While traditional OEMs are shifting resources to the software world, other players such as Tesla (my article on product development at Tesla) and Alphabet-owned Waymo start on a green field with a Silicon Valley mindset.

The goal of both approaches is to combine a physical product with digital software. The success of this "phygital" convergence thus becomes a decisive competitive factor. Both established automakers and (not so) new players must therefore wisely combine the ingredients for a reliable, functional and safe ACES ALM so that vehicles as a finished product are subsequently desirable for the end customer.

Challenges? Opportunities!

To make ACES ALM a success, automakers must therefore keep an eye on a wide variety of aspects and fields, which can be divided into four clusters.

• First, there are the ACES engineering business drivers, i.e. the fundamental technical interaction between software and hardware and their mutually dependent development including data-driven engineering.
• Second, the right processes and methods are needed to help with such development - from model-based systems engineering (MBSE) to variant and configuration management to scaled agile.
• Third, this technical-methodological complex is encompassed by regulatory requirements for hardware and software in vehicles, be it Automotive SPICE, cybersecurity or functional safety.
• Fourth and finally, there is alignment with the company-wide IT strategy.

I will discuss all four of these areas of challenge - as well as the opportunities they present - in the next article in this series. Stay tuned!

Note: this is a cross-post from my LinkeIn article

Product development at Tesla: Software

 

In December 2021, I published an initial article on Product development at Tesla. In this follow-up, we will look deeper into a domain which is a clear strength of Tesla: software. Although this attribution usually refers to software as part of the product, we will also look into the enterprise software which helps building these products.

Enterprise software @ Tesla

A CleanTechnica article cited Elon Musk as follows:

“We are not dependent on enterprise software. For those who understand, that is a very big deal. My hat off to the great work of the internal applications team. They write the nervous system, the operating system, the Tesla company operating system. Extremely fundamental!”

Most OEM’s use commercial-off-the-shelf software for ERP, CRM / Dealer Management etc. from vendors such as SAP, Oracle or Salesforce. But Tesla decided to develop custom solutions. Why? And how did they do it?

Jay Vijayan was CIO at Tesla from 2012 – 2016. Before that phase, he worked on ERP and CRM at Oracle and VMware. Even with this background, his first task at Tesla still seems like a Mission Impossible: building an ERP in three months. Leaving Tesla in 2016, he founded Tekion Corp. which focuses on enterprise software for the automotive industry, currently an SaaS solution for sales, finance and insurance.

Based on these facts, some speculations on the “Why” and “How” come to my mind: Elon Musk had a strong vision of Tesla as a software-driven company with high vertical integration. He probably also has foreseen the need for flexibility in the enterprise software due to major business model extensions such as charging networks, batteries, CO2 emission credits etc. On the other hand, Jay Vijayan knew how expensive and inflexible traditional enterprise software can be. I can only guess that some of his lessons learned at Tesla found their way into Tekion.

Product software @ Tesla : AI and FSD

On December 24, 2021, Tesla introduced Software V11.0. It brings new features such as the light show, games, adjustable subwoofer punch or blind spot visualization with the live camera view. That is certainly an added value for Tesla owners(read: justifies a higher price), and the over-the-air delivery of these updates is still a benchmark in the industry.

FSD (full self driving) is certainly a controversial feature, but it is essential for the data-driven business model of Tesla. Software plays a major role in improving this feature, Lex Fridman pointed out the importance of AI after the Tesla AI Day in August 2021.

Here are a few highlights from the Tesla AI Day 2021:

• In-house developed toolchain for working in  the “vector space” for debugging of the neural networks, labeling (automatic & manual) and neural net compilation. As Tesla is mainly relying on camera images as sensors, this approach allows to label objects once in 3D and project the labels on many image views.
• In-house data labeling team with 1.000 persons
• The neural network is fully retrained every 1-2 weeks with 10.000 GPU’s , which puts Tesla among the top 5 supercomputer companies in the world
• An AI evaluation infrastructure with 3.000+ FSD computers as hardware-in-the-loop in 3 datacenters + cloud with custom job scheduling & device management software
• The Unreal engine is used for basic physical simulation. Simulation is “a video game with Autopilot as the player“. Again, this is a controversial bet and a more deterministic approach is needed for safety. But this simulated environment with the 3D objects is great for labeling.
• Scalable scenario generation with hand-made, procedural and ML-based scenarios. This also includes a toolchain to replicate scenarios and environments anywhere a Tesla has driven.

Tesla seems to be pretty successful at DevOps with continuous integration and deployment for the full stack from ECU basic software over apps in the car incl. HMI and apps on a phone to backend software. Interestingly, the career website has 15 categories of jobs, but Engineering&IT is combined into one. The job descriptions reveal some details into software development at Tesla:

Excerpt of requirements for a “Full Stack Engineer”:

• Python development experience (2.x, 3.x)
• Experience with web technologies and web frameworks (React/Node/Dash/PHP/CSS, etc.)
• Experience with workflow management platforms (Airflow or similar)
• Basic knowledge of stream processing systems (Kafka, RabbitMQ, or similar)
• Experience with scalable map-reduce data processing preferred (Spark, Hadoop, or similar)
• Experience with at least one of MySQL/Postgres/NoSQL databases
• Experience with containerization (Docker or similar)
• Experience with build systems, automation, continuous integration (Rundeck, Jenkins or similar) 

I hope that this post helps to shed some light on the software development approach of Tesla. Please feel free to comment with any additional insights and links you can share.

Note: I published this article originally on LinkedIn.

Product development at Tesla

 

 

In December 2021, I read the special edition of Germany‘s Automobilwoche on „The Tesla Principle“ and was hoping to find some answers on the question "How does product development, systems engineering and PLM work at Tesla?". This article summarizes my findings including feedback I got on the original post. If you have additional insights to share, please feel free to comment here or in the original post.

The articles cover sales, marketing, production, Elon Musk etc. - but not product development. Even a Google search doesn‘t reveal much but some job postings from Tesla asking for skills in Dassault Systemes 3DEXPERIENCE, CATIA, ENOVIA, DELMIA, SIMULIA and Solidworks applications. The related press release from Dassault Systemes on this is from September 2010...

Although Tesla is very secretive about their way of doing things, the articles give some insights into product development at Tesla. Here are my takeaways:

1. Tesla designed their products from the ground up without restictions from existing platforms, production technology etc.. They especially had a Silicon Valley mindset with focus on software and E/E architecture. But they still needed to design and build a mechanical car around the software.

2. They have created an integrated System of Systems – from battery cells over charging infrastructure and powerwalls to actual power production. I am not sure if this is the result of great systems engineering, but it certainly turned out well.

3. Tesla has a culture of innovation rather than cost-cutting

4. Having a tech-savvy CEO and flat hierarchies certainly helps. According to Elon Musk „pace of innovation is all that matters in the long run“

Agile@Tesla

And then, there ist the agile way of working at Tesla that not only applies to product development, but to the whole company including suppliers. As pointed out by Oliver Wörl and documented by Joe Justice in several videos, one cannot underestimate the importance of agile at Tesla:

• Agile-native company with iterations of 3-6 hours and empowered people
• An error culture that is required to operate at this speed
• Data-driven processes - e.g. all cars providing data constantly on usage patterns, issues, preferences etc. Analytics is then applied to direct and prioritize the agile work.
• Focus on building cars, almost as if product development is not separate from manufacturing but part of the iterative processes. Small iterations are deployed immediately to production (CI / CD = continuous integration / deployment), i.e. a pretty direct integration between product development and manufacturing almost in an "ETO = engineering to order" model as opposed to the mass production model in other OEM's. This can imply individual homologation testing of each car at the end of the production process.
• No meetings, just online chat. And no Powerpoint presentations.

Note: I published this article originally on LinkedIn.