Arlene Minkiewicz is a software measurement expert for PRICE Systems who is dedicated to finding creative solutions focused on making software development professionals successful. We recently sat down with Arlene to learn the basics about cost estimation and predictive analytics.
Tell us a bit about your background. Why did you pursue a career in cost estimation and research?
I realized early on that math and science were my strong suit and my passion. In high school, I was enrolled in a special college prep program focusing on mathematics and engineering. I studied at Lehigh University and graduated with a BS in Electrical Engineering. My first was a typical job for a fresh-out-of-college grad: working for an organization that built nuclear power plants, which required lots of boring paperwork for the new guy (or girl in my case). I gave it a good year and then decided it was not for me.
What I had discovered during that time via various side jobs I took on was that I really enjoyed programming and did in fact excel in it. So I found a job with PRICE Systems developing software. I started working on internal software systems and quickly progressed to the development of the cost estimation software we sold to our customers.
While I enjoyed this, I was greatly intrigued by the the actual equations in the model and how they are developed. Additionally, being on the cost research side of the organization (rather than the programming side) offered significantly more opportunities to travel to customer sites and conferences. The best part of my job is the fact that I need to constantly track technology changes and provide guidance and equations to help my customers successfully estimate their complex technical projects and programs.
If someone were to say to you, “Cost management optimization is just a fancy term for finding ways to spend as little money as possible on a project or initiative,” how would you respond?
I would disagree vehemently! First of all, if you scour the news for project failures, particularly with software intensive systems, failure to estimate correctly is often cited as one of the top reasons for project failure. Cost management optimization seeks to provide project managers the tools they need to understand the implications of both overestimating and underestimating projects. Having said that, cost management optimization is truly a fancy phrase for the introduction of realism into the cost estimation and project planning exercise.
Is cost estimation more of an art form, or is it a skill that can be taught to anyone?
While good cost estimation definitely is supported by sound mathematical algorithms and processes, it is still considered an art form in many cases. When I interview potential candidates for cost research positions, I often describe what we do as 50% math and statistics and 50% private detective. Much of what we do from a model development perspective is understand technology and how we know it impacts costs, and then opine as to how future improvements will influence costs based on what we learn about the past and current states.
If you look at the estimation question from the perspective of our clients who need to use our tools to perform estimates, they need to be able to understand and quantify technologies and processes that they in some cases have not even invented yet. They need to understand technologies in their industries and translate that understanding to input parameters in their cost models. Not every estimation exercise is like this, but many require the application of knowledge and understanding, tempered with a dash of intuition, and always overseen by common sense.
When companies try to estimate the costs of a product, project, or solution, what do they often fail to take into account?
It definitely varies from estimator to estimator and from organization to organization. When people do a bottoms-up estimate, which is focused on understanding costs at the component level, they tend to underplay the activities around the integration of these components into a complete system. Seasoned cost estimators tend not to miss much as long as they are estimating within their wheelhouse. Once they start to estimate something outside of their comfort zone – especially if it’s new or unrealized technologies or processes – there is often a tendency to underestimate the costs of getting the technology mature enough to use or the learning curve associated with new processes.
People will assume that since they are using an agile development process for software that this will reduce their costs because they read somewhere that it was a more efficient way to do software. While this may or may not be true (there is evidence to support both sides of that story), if this is the first time someone is doing an agile project, it’s not going to be less expensive than their current model because there is learning that needs to occur.
What kinds of common corporate occurrences or actions tend to have the most drastic effect on the cost estimations of a product or solution?
Often times, the estimators go into an estimation exercise knowing what the “right” answer should be. In other words, they know what their managers expect them to come up with.
I had an experience once where, after being presented with a well-thought-out estimate with lots of history and data to back up its results, a project manager said “That estimate can’t be right. I only have 6 people, and we only have a year to do it.” “Wishing” that a project would only cost $500,000 is a bad way to estimate projects. Sometimes I hear statements such as, “I know it cost $500,000 last time, but we’re gonna put better people on it with state-of-the-art tools, so we can certainly cut the costs by 40%.” The proper response to that is, “Show me a study that proves that – one that does not come from the manufacturer of said ‘state-of-the-art’ tools.”
Another area where corporate behavior can influence the success of the cost estimate is when “requirements creep” is tolerated or even encouraged. If the project you deliver a year from now has 50% more functionality than the one you prepared an estimate for, that’s not a bad estimate; that’s poor management of the project and the customers. Estimation should not be a one-time exercise. Projects change and sometimes those changes are important and necessary. Good management recognizes that:
(1) if they add requirements, the last estimate is no longer valid,
(2) If they don’t do this, they will finish late and over budget, and
(3) if schedule and budget are not negotiable, adding requirements should be accompanied by the removal of requirements of the same size and complexity.
Finally, another area where corporations display bad behavior is to take the first number they get – the one they asked the development team to give them (just a ROM) when there is little known about the projects – and never let that number go.
To what extent is predictive analytics an exact science?
Predictive analytics, especially in its applications to cost estimation, is far from an exact science. If one were to develop a predictive analytics model intended to model very rational and well-defined behavior, one could claim in that domain that predictive analytics approaches exact science. In the cost world, it will never happen.
Cost data is some of the noisiest data I have had the pleasure to work with (and it is in fact a pleasure – one can learn even from the noisiest data if one thinks outside of the box). There are several reasons for this. At the end of a project, you can tell how many lines of code you’ve written, and you can (with some expertise) effectively quantify complexity and team experience on a well-defined scale.
The problem often comes with how much actual effort or money was associated with the program or project. There are several reasons for this. You may be asking people to provide quantitative measures intended to understand how much work they do and effectively measuring their productivity. Not everyone is totally comfortable with that process, and they may cook the books. There are also sometimes political or customer relations issues that require an organization to under-report or over-report how much something cost.
We are constantly trying to find public sources of cost data that we can share with our users to offer them guidance on their estimates. It is often possible to find unit production costs for equipment that our customers often estimate. So I can find out what it costs (within a relatively decent margin) to produce the F-35; but if I want to understand what it costs to develop the F-35, or to develop the software that runs its various systems, that data is a lot more elusive. Another factor that complicates is the fact that organizations will often invest internally to grow a technology in order to win a contract. This data is not accounted to the project but rather to the IRAD monies.
Here’s where I see the real power of predictive analytics for our customers. We are currently involved in several engagements where we are providing tools and mentoring to help our customers grow internal data collection and analysis centers where they can use predictive analytics to grow cost estimation successes with their projects
If someone wanted to take courses to learn how to work in cost estimation or predictive analytics, what core skills and areas of knowledge should they already have in order to be prepared for the coursework?
Clearly, math and statistics are important skill sets to build cost models or to use industry tools effectively. I think it’s also important to have a good background in engineering, software, manufacturing processes, etc. depending on the types of estimation (or estimation models) you expect to do. So if you’re going to work for Boeing as an estimator or model builder, having some knowledge of avionics and composite materials would facilitate the conversations that you as the estimator may need to have with the engineering staff.
When I look for new cost research analysts for my group, I find that systems engineers and industrial engineers offer a great broad-brush knowledge that makes a good base for the kinds of studies we have to do. In some organizations, the people they hire for estimation have mostly a financial background. While there are aspects of a financial background that are important to estimation, for the kind of estimation that our clients usually do, having a technical background is often key.
With the growing capabilities of artificial intelligence, how will it impact cost estimation and/or predictive analytics over the next ten years?
Technology increases such as artificial intelligence, cloud computing, and big data are all phenomena that will offer great opportunities to help estimators better translate what they learn from historical projects into knowledge that’s vital for successful estimation of future projects. Not that they will make predicting an uncertain future an exact science; there will always be uncertainty around cost estimates for lots of reasons. Humans will still be optimistic and driven by motivations other than a singular quest for the truth, things happen in projects that impact costs that are impossible to predict, brand new technologies can’t be fully understood based on an examination of older technologies, and the list goes on.
Basically, technology advances will absolutely improve the ability of cost estimators to learn from their past. But will artificial intelligence techniques help us to nail the cost of the next advances in artificial technology, especially the ones we haven’t conceived yet? Maybe they’ll help us get closer.
Thinking about exploring cost estimation, predictive analytics, or another type of occupation? View our open courses today!