Computational creativity

As part of its quest to develop cognitive systems, IBM Research is exploring whether a computer can be creative by designing a machine that can create surprising yet flavorful recipe ideas no cookbook has ever thought of in order to enhance human creativity.

Share this page

What is computational creativity?

Computational creativity applies technology to assist humans in thinking outside the box and expanding their exploration boundaries.

In this instance, IBM researchers have built a system that can reason about flavor the same way a person uses their palate. While the original Watson Deep QA technology that debuted on Jeopardy! demonstrated how a system can learn to reason about the world as it is, IBM Research is exploring the next step of a machine’s ability to model human intelligence: generating ideas the world has never been imagined before. This is computational creativity.

Transforming how food is created

Computational creativity has the potential to radically transform the food industry in ways we have never seen before. Today, a significant portion of a food services or manufacturers’ business is focused on generating new ingredient combinations and finding new flavors that will be a commercial success. Data is the secret ingredient in these companies’ quest to perfect a new recipe. This research will allow chefs and other food professionals to be more creative and enable consumer packaged good companies to shorten the time to bring a new flavor to market by making the development process more efficient.

Having an impact

While IBM is using the domain of food to explore and test its research, there’s an opportunity to make a societal impact and tackle the challenges of obesity, malnutrition and hunger. Food manufacturers, school lunch providers and chefs all strive to create foods that satisfy people’s various tastes and preferences, but it’s a challenge to also make them healthy, rich in nutrients and adhere to different dietary standards. By using computational creativity technologies to analyze the chemical compounds and ingredients, food professionals can identify new recipes and pairings that are not only tasty and healthy, but also efficient to produce.

The science behind computational creativity

Creating a recipe for a novel and flavorful meal is the result of a system that generates millions of ideas out of the quintillions of possibilities, and then predicts which ones are the most surprising and pleasant, applying big data in new ways.

The system begins by capturing tens of thousands of existing recipes through natural language processing techniques to understand ingredient pairings, ingredient-cuisine pairings and dish composition, which it rearranges and redesigns into new recipes. It then cross references these with data on the chemistry of food ingredients, and the psychology of people’s likes and dislikes to model how the human palate might respond to different combinations of flavors.

A key component of the system is generating something that will be perceived as surprising. IBM Research is exploring this through information-theoretic notions of surprise. We have prior beliefs of what food is—based on our cultural, historical and personal experiences—and when we introduce a new idea or food it will change our beliefs. The greater the change in beliefs, the more surprising. For example a meal of bear meat and sandalwood is more likely to rank high in novelty over a more common set of ingredients because it is not a dish that is current in our culture.



Meet the researchers

  • Florian Pinel

    Florian Pinel

    Senior Software Engineer - Service Ecosystems and Computational Creativity,
    Thomas J. Watson Research Center

  • Anshul Sheopuri

    Anshul Sheopuri

    Manager and IBM Master Inventor,
    Thomas J. Watson Research Center

Beyond food: Computational creativity and the future of customer experience

A system that can model human intelligence and generate new ideas has many applications outside of food and the opportunity to transform customer experience. Truly superior customer experiences are based on perception—appealing taste, appearance and design, to name a few—and represent a major differentiator in a variety of industries, including retail, consumer goods, hospitality and travel.

So are creativity and customer experience intertwined—a dish should taste delicious, a new fashion accessory should be appealing, a commercial should be engaging. What we remember is the experience, and computational creativity is an approach for helping differentiate experiences. As companies race to bring new products to market to remain competitive, computational creativity can help design what differentiating features should be prioritized for a new or existing product. The technology can also help consumers purchase products. A fashion company can use knowledge of a consumer’s existing clothes and accessories, current themes in fashion and the brand’s inventory to provide the consumer with various combinations of clothing for a new look.

IBM is already working with customers on applying computational creativity technology to their business.

-Mahmoud Naghshineh, Vice President, Services research, IBM Research

To learn more about this research and how it can transform your business, contact Research Staff Member and Manager of Consumer Modeling Anshul Sheopuri.


Real time collaboration and recipes

IBM is collaborating with the Institute of Culinary Education (ICE) of New York and its team of world-class chefs to codify culinary knowledge into mathematical terms a computer can understand, and provide feedback on the system’s suggested recipes. An important aspect of this collaboration is pairing human creativity with machine creativity to create the best possible outcomes and results.  This is being illustrated by blending creative and flavorful ingredient combinations suggested by the machine, with creative interpretations of how to prepare and cook the dish suggested by ICE’s professional chefs.

IBM and the ICE team taste tested a variety of computer generated recipes, including some of the following. (Click on the + sign to expand each recipe below)

Caymanian Plantain Dessert

Yields 6 servings
Special tableware: 5 cm diameter, 7.5 cm tall glassescaymanian plantain dessert

Caramelized bananas

21 g butter
28 g molasses
1 tsp (5 g) pure vanilla extract
about 0.3 g nutmeg
170 g peeled very ripe bananas, medium dice (1.25 cm)
85 g milk

  • Heat the butter and molasses in a saucepan over medium heat.
  • Add the vanilla extract and nutmeg, then the bananas, and cook for 2 minutes, stirring regularly with a spatula.
  • Add the milk, stir, and bring to a simmer. Remove from the heat. Adjust the nutmeg as needed: you should be able to taste just a hint of it.
  • Pass the mixture through a sieve. Process half of the bananas with the liquid in a blender until smooth. Transfer to a container, mix in the rest of the banana chunks, and let cool for 30 minutes.
  • Pour into the verrines, and refrigerate for at least 30 minutes. Once cold, the mixture should not be liquid anymore.

Coconut and lime pastry cream

3 egg yolks
45 g light brown sugar
14 g flour
170 g milk
17 g lime juice
28 g coconut flakes
3 g butter, diced

  • In a bowl, mix the egg yolks and half of the sugar with a whisk for 1-2 minutes, then mix in the sifted flour.
  • In a small saucepan over high heat, place the milk, the lime juice, coconut flakes and the rest of the sugar, and bring to a simmer. Remove from the heat and let steep 5 minutes.
  • Process the milk mixture in a blender, and pass through a conical sieve, pressing with a ladle to get all the liquid out of the coconut residue. Return the liquid to the saucepan, and bring back to a simmer.
  • Slowly pour the milk over the egg yolk mixture to temper it, whisking constantly. Return to the saucepan, and bubble gently for 2 minutes, still whisking constantly. Transfer to a container, mix in the butter, and let cool for 15 minutes.
  • Pour into the verrines, and refrigerate for at least 30 minutes. Once cold, the mixture should not be liquid anymore.

Papaya and orange salad

113 g orange juice
20 g butter, diced
about 0.1 g cayenne pepper
128 g papaya, small dice (1/4”)

  • In a saucepan over high heat, reduce the orange juice to ¼.
  • Whisk in the butter and cayenne pepper. Adjust the pepper as needed: you should be able to taste just a hint of it.
  • Toss the papaya, and remove from the heat. Transfer to a container, and let cool for 15 minutes.
  • Pour into the verrines, and refrigerate for at least 30 minutes.

Plantain chips

Corn oil, for deep-frying
1 plantain, very cold
salt

  • In a deep-fryer, heat the oil in to 375 F / 190 C.
  • Peel the plantain, then cut it in half. Using a mandoline, slice each half very thinly: the slices should be just thick enough so they don’t break. Cut each slice in half lengthwise (into 2 long strips).
  • Proceeding in small batched, deep-fry the plantain strips until golden brown, then drain on paper towels and let cool.
  • Deep-fry the strips a second time for about 10 seconds, to make them crispy. Drain on a paper towel, and season with Salt.
  • Place 2 chips on each verrine just before serving.

Creole Shrimp Dumpling

Yields about 40 dumplingscreole shrimp dumpling

Creole filling

35 g butter
160 g ground lamb
1.2 g = 1/2 tsp ground black pepper
3.5 g salt
60 g celery, small dice
110 g canned tomato chunks, small dice
200 g broth (chicken stock)
100 g okra
160 g shrimp, small dice
1.25 g = 3/4 tsp file powder
5 g chopped parsley
15 g lemon juice

  • Cut the okra into small dice, then blanch and let cool.
  • In a pan over medium-high heat, sauté the ground lamb in the butter. Add half of the salt and black pepper, and cook until well done, stirring regularly. Add the celery, and cook for a few minutes, until soft.
  • Add the tomatoes, broth, okra, shrimp, file powder, and the rest of the salt and black pepper, and then simmer over low heat until reduced. Don’t rush it. Let cool, and refrigerate.
  • Mix in the parsley and lemon juice.

Assembly

40 wonton wrappers (or pasta dough made with chickpea flour)
creole filling (see above)
canola oil (for deep-frying)

  • Proceeding in batches, place a spoonful of creole filling in the center of each wonton wrapper (don’t overfill), fold in half, and use water to seal the edges. Store the dumplings on a sheet tray dusted with flour and wrapped in plastic film until ready to cook.
  • Cook the dumplings in salted boiling water until soft, then drain.
  • Deep-fry the dumplings at 375 F until golden brown, drain on paper towels, and serve.

Swiss-Thai Asparagus Quiche

Yields 14 quiches
Special equipment: Tart molds, 2" diameter, 1" tall

​Pastry dough

swiss thai asparagus quiche

8 oz sifted AP flour
1 tsp salt
1 egg yolk
2.5 oz water
4 oz butter, diced at room temperature

  • Place the flour in the bowl of an electric mixer fit with the paddle attachment. Add the salt, egg yolk, and water, and mix over low speed. Add the butter, and keep mixing until homogeneous. Knead the dough by hand for 1 minute.
  • Shape into a ball, cover with plastic wrap, and refrigerate for at least 1 hour.

Quiche

Pastry dough (see above)
0.4 oz lemongrass (peeled, tender white part only)
4 oz leeks, white part only
1/2 tsp salt (shared between the ingredients as described below)
1.5 oz butter
28 asparagus tips, 2.5" long
0.4 oz water
3 eggs
1 egg yolk
7.5 oz heavy cream
7.5 oz plain whole-milk yogurt
3/8 tsp mild curry powder
1/4 tsp ground coriander seeds
1/8 tsp ground black pepper
2 oz feta, crumbled
1.5 oz gruyere, grated
1/2 tbsp finely chopped curly parsley

  • Lightly grease the tart molds.
  • Roll the pastry dough to 1/16" thick (this gave me a rectangle of approximately 14" x 16"). Cut 14 discs of 4" diameter, and fit them into the tart molds. Refrigerate for 30 minutes.
  • Slice the lemongrass and leeks very thinly. Melt 2/3 of the butter in a saucepan over medium heat, then add the lemongrass and leeks, season with salt, and cook until soft, stirring regularly. Let cool.
  • Melt the rest of the butter in a saucepan over medium heat, then add the water and the asparagus, season with salt, and cook for a couple minutes. The asparagus must still be crunchy. Let cool.
  • In a bowl, mix the eggs, egg yolk, heavy cream, curry, coriander, pepper and the rest of the salt. Fold in the yogurt, lemongrass, leek, and crumbled feta.
  • Pour the quiche mixture into the molds, arrange 2 asparagus tips on top of each of them, and sprinkle with the grated gruyere. Bake in a 400 F oven for about 30 minutes, until golden brown.
  • Serve warm with some parsley sprinkled on top.

Turkish Bruschetta

Yields about 24 servingsTurkish Bruschetta

Eggplant purée

1000 g Japanese eggplant (about 3)
8 g sumac
1 g dry oregano
parmesan cheese
10 g sunflower oil
6 g basil, chiffonade
4 g salt

  • Char the eggplants on a flame until black on all sides, then roast in a 350 F oven until tender.
  • Scoop the flesh out with a spoon, keeping only a little bit of charred skin. You should have about 450 g of flesh. Add the sumac, dry oregano and parmesan.
  • Sauté the mixture in oil over very high heat for about 30 seconds, stirring constantly, then transfer to a bowl, add the basil and salt, and let cool.

Carrot mixture

100 g coarsely grated carrot
28 g scallion, tops removed, sliced on a bias
0.75 g cumin
1 g Hungarian sweet paprika
2 g sumac
2 g salt

  • Place the carrot, scallion, cumin, paprika, sumac, and salt in a bowl. Toss and reserve for at least 30 minutes.

Assembly

24 thin slices of baguette
sunflower oil
eggplant purée (see above)
carrot mixture (see above)

  • Drizzle some sunflower oil over the bread, and toast until crispy.
  • Spread the eggplant purée on the bread slices, then top with some of the carrot mixture.

Smart Machines: IBM's Watson and the Era of Cognitive Computiing

By John E. Kelly III and Steve Hamm

Share this page