The Big Blue’s Way of Learning Engineering – IBM

by WiseOcean | Sep 14, 2020 | AI Digital Transformation, Corporate L&D, Talent Digitalization

IBM has been a big blue giant and an AI leader in our memory, its Deep Blue won over the human master at chess in 1997, its Watson won Jeopardy! in 2011, its Project Debater can debate with humans on complex topics. Now IBM’s AI technology has been used in many tough quests from picking stocks to knowledge discovery. How does the Big Blue use AI to transform itself and keep its competitive edge? It all started with transforming its people with personalized development at scale, which isn’t possible without Learning Engineering constructed by experts then automated by AI. Vinod Uniyal, Chief Technology Officer (CTO) of Talent & Transformation at IBM, had a chat with us to share how it works under the hood.

Highlighted quotes:

Five to seven years ago, when we started looking at new technology areas such as cloud, data science, artificial intelligence in a much more meaningful way. That is when we started our transformation, that transformation meant not only changing the way we work, the culture of the organization, the structure of the organization, but also, you know, how do we develop the talent and the skills that we needed for this transformation to be successful. … we also tell our clients that the starting point is understanding the skills that you have.

I already talked about the skills inference, understanding your current skills, then the second step is understanding future skills.

Think about the ways in which you would probably want to ask people — give me what skills do you have, give them a form and ask them “do you have this skill, that skill” — It doesn’t work, it’s a broken process, the only way we can make it work is to automate it by using AI and continuously training. Those algorithms over a period of time will become automated processes, and then it is available all the time. That’s the only way it can work at scale.

There is a big transformation that needs to be made to get to where we are now. The starting point is to build digital literacy, not only in L&D department but also in the entire organization. So everyone has a basic understanding of digital transformation, digital skills, the value of data, how we can leverage data. Having that, the adoption becomes easier. At the same time, the L&D and HR departments look very different today, and they are supposed to look different. The organizations that want to leverage data, AI and all these technologies will need to have certain data skills internally although they can work with service providers.

Jessie: Could you introduce yourself and your role at IBM to us?

My role at IBM is to lead technology for talent development practice, not only for learning and development but also the skills transformation, working at the intersection of AI, Cloud Technologies and Talent. I am very passionate about what I am doing. I am responsible for creating the offering, the Talent Platform and the Ecosystem of IBM learning and skilling solutions for clients and I also work with the clients. I also work with IBM internal HR service team very closely because the solutions we have developed and are being used within IBM, and we take the same solutions to the market. I’ve led the commercialization of many internal solutions. For example, “IBM Your Learning” is a learning SaaS multi-tenant platform where IBM employees go to learn, it was developed for our employees first and then extended to the public and commercial clients later. 

We actually see IBM as our first client when we develop learning solutions. I would say that helps us build solutions from a  customer-centric mindset, and not to see it as an internal thing. Even our internal employees are our customers. I would like to add that we’ve had many people in our internal IBM HR organization who have actually moved into our consulting practice and the other way around also, so we know exactly what is going on (emerging best practices) in IBM HR. 

Jessie: Could you talk about the L&D digital transformation history at IBM? 

IBM is almost a 109-year-old company by now. It is still surviving and thriving because IBM has transformed itself over and over again. But if you just talk about the last significant transformation, I think it happened about five to seven years ago, when we started looking at new technology areas such as cloud, data science, artificial intelligence in a much more meaningful way. That is when we started our transformation, that transformation meant not only changing the way we work, the culture of the organization, the structure of the organization, but also, you know, how do we develop the talent and the skills that we needed for this transformation to be successful.

And there have been a lot of strategies that we have used in order to accomplish that kind of skills transformation. I’ll talk about some of those strategies here. So one of the key things there is to understand the skills that we have currently. And this is what we also tell our clients that the starting point is understanding the skills that you have.

Lot of surveys that we have done with the chief executives, most of the chief executives tell us that they really do not know what skills they currently have in their organization. They also struggle in saying that they have all the skills that they need in order to meet their strategic business objectives.

In IBM, since it is such a large scale, you know, we have ~360 thousand employees, what we have done internally is we use skills inference to understand the current skills. What I mean by skills inference is using the digital footprint of the employees to infer the skills from the data, right. So, we use more than 26 data sources as of today, which are internal data sources. Think of your internal intranet, like the collaboration portal where we are posting something, the places where we put our delivery, our internal performance management, those kinds of things where some sort of contributions or deliverables are shared by people, and of course, the training — the kind of training and coaching somebody is undergoing. 

So we look at all these data sources and try to infer the skills of the people. So once that has been done, then we present these skills in terms of a skills profile to the employee, we show them that these are the current skills that you have. 

Now the second step in skills transformation is to understand the skills that you need for the future. And then the third step is to close that skills gap. And I’ll just repeat it so that we make it a very simple process.

I already talked about the skills inference, understanding your current skills, then the second step is understanding future skills. That is something you can do based on your strategy, interviewing the business leaders, looking at what the trends are, where you want to go and all that. So for IBM, these skills were, probably Cloud, Analytics, AI, Data Science, on the hard skills side, then there are certain ways of working or soft skills, such as Enterprise Design Thinking, Agile, and all other skills like Negotiation, Effective Communication, etc. So basically, we went into certain technology areas and also changed the way we work. You know, how do you work, and for us enterprise design thinking and agile is something which is ingrained into everything we do, including the learning and development department as well. That’s a big change for such a large organization. IBM was supposed to be this big Elephant, the Big Blue, and supposed to move slowly. But in the last five years, I’ve seen tremendous change in IBM. 

So, remote work culture, the agile working culture is something that is ingrained. Now, one of the key aspects is once you know that these are the skills that we need, these are the job roles that we need more in the future, you also need to transparently communicate this to your employees, right? So, transparency is very, very important. And I would say once you have that transparency it builds trust, so employees know and understand what the organization actually needs, they will be wanting to learn more, they will say “I want to become a data scientist” or “I want to learn AI or Analytics” or something else…and that has happened in IBM. So, once you have transparently communicated the hot skills and the hot roles, then you should help the employees to understand their skill gaps. 

I personally believe in super personalization. The learning that you have available should be personalized to you so that it is relevant for you. Think about Netflix as an example where you get personalized recommendations and you just get hooked on to it. So, similarly learning for us works in the same manner, so, what you are interested in, what people like you are interested in, that is what gets recommended to you. Those AI based personal recommendations actually work like magic in creating that engagement, super engagement in terms of learning. 

And I would say it is not only limited to learning, but it goes beyond that because learning doesn’t happen only by consuming digital content, you need to also share it with others, practice what you’ve learned, and then have opportunities to perform tasks and use those skills. So all these elements have to come into play for you to really build those skills. 

So that means: learn, share it with others, get coaching from people (like people who are experts in those skills that you are interested in) then get certain opportunities, which we call as internal gig economy so that you can practice those skills in those ad-hoc projects. And once you build those skills, then you get personalized recommendations for jobs which are available internally, so that you can actually move into those different roles. 

So this JML (Join-Move-Leave) employee lifecycle, I’ve expanded the cycle to something which I call join, learn, share, grow, and then move internally. That’s the entire life cycle of talent development and skills transformation. 

Jessie: Thanks for giving us a whole picture, and simplifying the steps, but the most difficult is the skill modeling. How does IBM do skill inferencing? How to verify the inference? What is IBM Talent Framework? How is it built? How does it work in IBM’s learning ecosystem and support personalized learning? How should an organization use it?

Yeah, I made it very simple. But it is not, the devil is in the details. You’re absolutely right. There are two parts to it. The first step is understanding the skills, you definitely need a base, you need to have a definition of skills, which we call the taxonomy of job skills and roles. And if you have a library, then you get a jumpstart rather than having to build that jobs and skills taxonomy on your own. And we have a library in IBM, which we call IBM Watson Talent Frameworks. So it’s a job and roles taxonomy spans around 16 different industries. 2000 plus skills and thousands of job roles are already defined in that. It is something that we have curated over the last 30 years or so and we continuously keep it updated. So if you start with something like that, you have a jump start. And we can actually help you with that definition of job roles and skills (for your business needs). 

And then you can use this definition to infer the skills from the digital footprint data that you have, and match it to the employees. So that will form the base. And once you have it, then you can use these skills to do a lot of other things. And I’ll give you a couple of examples. 

One of the things that you can do is tag the learning content that you have with these skills. So if you’re learning content is tagged with the skills then it becomes easier for you to recommend those learning to those people who are interested in jobs linked with those particular skills. And similarly, you can do it with the job roles so you can tag the learning content with the job. So people who are interested in those job roles can get recommendations for that learning content. 

Jessie: There’s a crucial link here — you talk about the definition, that’s human languages, how about the machine side? When the machine processes those data, the data from the learning records and also from work behaviors, how do those data represent what skills a person has? There is also an assessment layer — what kind of behavior represents what skills you have.

I can give you some examples that will make it a little bit clearer. So let’s say you have some cases with structured data, most of the cases are unstructured data. One example is you have undergone certain training in AI or Data Science, then you must have some sort of AI or Data Science skills at least foundational skills (according to your learning records). Another example is if you are a salesperson, we look at your sales pipeline and you if have 10 different opportunities pertaining to AI-related solutions, then we can infer that the person who has built that sales pipeline of AI-related solutions must have some sort of AI skills. 

The second question is the depth of that skill or the proficiency level of that skill. Most of these things initially would indicate only a foundation level of skill. So if you are making certain contributions to your internal website, you’re sharing something you have created, certain deliverables, that you have shared over a period of time, that will add to your proficiency. 

I’ll give you my own example, right. So in the last three months, I have conducted about seven internal sales enablement sessions, which pertain to our Talent Development Offering and Solutions, learning and skills transformation-related solutions that we have. All those sessions were recorded, published, they are available in our learning systems, so people can search for them. So, if I go to my internal website and I look at a solution called Expert Finder. It will identify me as a person who definitely knows something about Talent Development, and AI in Talent, according to his contributions. 

So first when you have all the data, you can bring AI algorithms, but then human experts the data scientists will have to get involved, you train the algorithm and make it work in your organization. Once you’ve trained it, then I think you can easily expect 70% ~ 80% kind of accuracy when you run this algorithm at scale. So I’m not saying this is very simple, but it is doable. And when you do it at scale, the benefits are obvious, you don’t want to go back to manual ways (of understanding skills). 

Think about the ways in which you would probably want to ask people — give me what skills do you have, give them a form and ask them “do you have this skill, that skill” — It doesn’t work, it’s a broken process, the only way we can make it work is to automate it by using AI and continuously training. Those algorithms over a period of time will become automated processes, and then it is available all the time. That’s the only way it can work at scale.

Brandt: I like your example of taking a presentation or demonstration, which may be harder to track and to account, but digitization makes it easier to track, not only makes it easier to find but, you can track how many people actually find it and use it to give you credit for that expertise as well as for sharing that expertise.

That’s absolutely true. So if you have conducted a session, that record is there, as well as how many people use it, how many people actually like it (like five stars or three stars), all these kinds of ratings are there. 

What we have also done is we have created certain Watson based algorithms, which we actually use in content curation, and content optimization. And what I mean by that is… so the content that we have in learning is in different formats, right? It could be SCORM based content, it could be Word, PDF, PowerPoint, etc.. The algorithms actually help us TAG this content automatically. As learning professionals, the instructional designers are usually Tasked with this (tagging of the content) — okay, you have created this content, put certain tags into it so that somebody can find this content. People would generally put 2-3 mandatory tags and be done with it. Does it make it discoverable? It does not. So the algorithm actually reads the content, almost as if a human being is reading it. And it brings out the key phrases from the content and other tags. And once we put this into the systems, it makes the content discoverable. It makes it personalized because now those key phrases and tags are available for any type of content that you have. So that is one example.

The second thing that we do with the same solution is to break the content into smaller parts. We call it chunking — chunking based on the logical composition of the content, we can break the content in terms of the syntactic structure of the content, or we can look at the semantic attributes as well. So syntactic is easier because you have a structure (e.g. Table of contents) already in place. But if there is no structure there, no table of content, it can read the content. And as soon as the topic changes, it breaks it there into a different segment (chunk). We have seen accuracy levels of almost 80 to 90% in these cases, which is fascinating. And then we expanded the same algorithm to do a lot of other things that I was talking about, — tagging the learning content with those skills, tagging the learning content with jobs, and job roles. 

Based on the jobs and skill taxonomy you run all other things. In most enterprises, it is a mess, right? They don’t have good data there. You may have a lot of jobs that are duplicate, which actually are similar jobs but have different names, jobs requiring similar skills. Unless you can clean that data, curate that data, you will not be able to use it. So, what we do with these AI algorithms is look at the jobs and the definitions, and the algorithms actually tell us what is the match between these two different job roles or 10 different job roles. That actually helps the human beings who are working on it. Now they can see if this job role and that role is like 90% similar, and ask “can we get rid of one of them?” So, that’s the way we are using AI to augment human consultants.

I can talk a bit about our strategic workforce planning approach. There are seven B’s. 

• The first is the Business strategy which links to talent demands. 
• For the Talent Acquisition, you can either Buy permanent resources, or you can Borrow contingent resources (temporary resources). 
• For the Talent Evolution, you can Bind the resources — that means move the resources from one part to another and retain them. 
• For the Talent Management, you can Bounce — meaning exit the resources that you don’t need, and Build — develop the resources. 
• For the Talent Augmentation, you can use Bots – this is very important, we think of Bots as resources whether it is automation or AI.
• Seventh B is Balance, continuously balancing your workforce rather than keeping it static and at most an annual planning exercise. 

This is our thinking about strategic workforce planning, this is how you can build an agile workforce that will continuously help you align with the changing external environment.

We have the project and learning marketplace, it’s sort of a talent marketplace within the organization where we have talent at one end (their skills), and then the opportunities at the other end — whether they are temporary opportunities or permanent opportunities. So there’s a continuous matching of talent with those opportunities and coaching opportunities, project opportunities, permanent job opportunities, all that happening on a continuous basis. That’s what I call a talent marketplace. It also doubles up as the place for you to crowdsource, you know, talent and innovation. If you have challenges that you want to throw at your people, it is the way in which you can actually build teams on an ongoing basis. You don’t have to have permanent roles to fill those projects, you can create projects and find the talent, which probably is residing in some other departments and build those agile project teams to get things done and then move on. 

Cognitive chatbots are built into the entire ecosystem. So, there you can talk to a chatbot and ask about the learning recommendations, you can ask any questions that are linked to learning or career or skills and all that.

We use AI algorithms for content curation, content personalization, I’ve talked about them. Content harvesting is again a very interesting application. If you have a lot of Informal sources within the organization, you can actually use the algorithms to source the content or harvest the content. You give these terms, objectives to the algorithms that I want to find content on, let’s say data science, it’s gonna find the content on data science from all the different sources that you will assign to it and bring it in front of you. So instead of human beings doing that kind of harvesting, we can use algorithms to do that content harvesting. 

About the credentials, there is a combination of open digital badges. In IBM, digital badges have been such a tremendous success in the last couple of years. The numbers are mind-boggling, actually. In 2019, IBM has consumed 27 million hours of learning and issued three million badges, and in recent weeks, we have been issuing almost 80,000 badges every week. The numbers have gone up drastically during the pandemic. We have our technology behind the digital badges to issue them automatically.

We are currently working with several institutions to set up Blockchain-based credentials. 

Jessie: Many enterprises don’t have good data so they hesitate to pursue personalized learning, what will you suggest to them to get started? What skill do the L&D department, business managers, and employees need to use IBM’s “cognitive learning” solution and those analytics insights?

There is a big transformation that needs to be made to get to where we are now. The starting point is to build digital literacy, not only in L&D department but also in the entire organization. So everyone has a basic understanding of digital transformation, digital skills, the value of data, how we can leverage data. Having that, the adoption becomes easier. At the same time, the L&D and HR departments look very different today, and they are supposed to look different. The organizations that want to leverage data, AI and all these technologies will need to have certain data skills internally although they can work with service providers.

Related References

Rebooting work for a digital gap

In addition to the new approach to performance management, talent management at IBM now includes a personalized learning platform and a personalized digital career adviser.

The platforms use data to infer which skills employees have and connect them with learning to build those skills that are increasingly in demand.

More predictive and prescriptive insights will be transmitted directly to managers and employees at the moment they’re needed most, embedded in the workflow.

(Jessie: the way IBM has engaged its employees to participate in the transformation journey with their voices isn’t possible without the help of AI, for example, Watson text analytics)

The enterprise guide to closing skills gap

IBM Talent Framework

IBM Skills (free resources for learning tech skills and for educators)

IBM awards its three millionth digital badge (and disrupts the labor market)