DigitalFUTURES Young : Theory of AI & Creativity

no three two one uh hello and welcome today to a special digital uh digital futures young event on theory of ai and creativity victoria luisa barbo and i gustavo rincon will be facilitating the dialogue we now introduce our selected presenters alexandra carlson university of michigan ann arbor weilong liao and yamin chen florida atlantic university gabrielle mira from the university of melbourne martin calvino independent artist the current team uh provides uh eli yah and our team from strelka institute of media architecture and design and michaela pinna from york university toronto today we have two a special two distinguished moderators matthias del campo and danielle bologna to share their insights observations and thoughts on today's presentations and now a brief introduction to dr matthias del campo dr matthias del campo a registered architect and designer is an associate professor of architecture at the university of michigan tubman college of architecture and urban planning is also a principal at span consisting of dr del campo and dr sandra meninger as a team a brief introduction to daniel bolajun is an assistant professor focusing on the application of computational design and deep learning strategies and architecture and architectural design processes daniel is a junior associate at kupema blau where he has a leading role of developing custom computational tools computational design strategies virtual and augmented realities applications as well as robotic fabrication processes he is currently a phd candidate at the university of applied arts institute of architecture vienna austria under the supervision of patrick schumacher i wanted to remind everyone about the call for papers digital's futures young sea level and climate change we are reaching out to our community for the submission of students and young research projects for this session sea level and climate change please submit for the deadline to be submitted is the 27th of march today's presentation will be as follows victoria will be introducing the panel today with a brief contextual summary uh we will then go to each of our individual panelists for eight minute presentations and then we will proceed to our discussion with our distinguished moderators our objective here is to have a dialogue that enables new shared insights and knowledge before we start the digital futures team i wanted to thank victoria her instincts and positive spirit inspires our research group i also will do a brief introduction my colleague victoria is a principal of resident matter a multi-disciplinary practice focused on the intersection between natural built and digital environments victoria is a designer an independent researcher interested in ideas surrounding the cultural and spatial effects of perception and ecology and now victoria thank you gustavo and yeah gustavo is a phd candidate and just finished his phd actually he has inspired um digital futures with all his energy and his intellect so uh thank you gustavo um so i wanted to kind of give a brief introduction before we start with our excellent seven presentations that we have today so we have a lot to go through today um but i kind of wanted to contextualize it and this is also following the previous keynote about ai and creativity that was taking place two weeks ago and we are very excited to have you know more contributions today by our scholars that submitted and are on this panel today so maybe you can go to the next slide um so a brief introduction every act of perception is to some degree an act of creation and every act of memory is to some degree an act of imagination this is what oliver sacks writes our bodies and minds are engulfed in a tangled multitude of sensory information from the environment hitting our cognitive apparatus we are not just consuming this information our brains are helping us build unique stories from the fragments that enter our consciousness our brains create and curate the world we live in computational landscapes are ever expanding computers have grown from the status of enigmatic machines of wonder to that of companions that are curating our lived reality our umbit being digitally connected metaphorically and literally wired to our devices has become the unique and ubiquitous state of being human-made technological objects are constantly evolving and are taking on new forms entangled in a complex feedback loop with the environments they emerge from they have large impacts on human lived reality technical objects in the digital age of machine learning are no exception humans have embarked on the quest for creating artificial intelligence a quest for not only creating but also understanding biological and more specifically human intelligence itself so what does it mean to be human in a creative universe what does it mean to be human in an age of artificial intelligence and what will the role of design and architecture be in an age of artificial intelligence or in the words of gregory bateson and that is what i want to leave everyone with before we embark on our endeavor with seven presentations today what is the pattern that connects so with no further ado i want to hand it over to our presenters our first presenter um will be where where is he um anyways so uh yeah please give it up to our presenters i'll give it over to you yeah thank you for the introduction i'm just having uh some issues with sharing the screen for some reason also i i would like to ask all the other presenters to um to switch off their videos while the others are presenting to kind of not overload the bandwidth if that's possible right thank you for the introduction victoria so my name is gabriel mira i'm a phd student at the university of melbourne and the title of my presentation is expertise playfulness and analogical reasoning three strategies to implement ai in architecture so in the past five years i've been working a lot with computational design and applied it to solve a variety of design problems this includes relation and geometric problems structural optimization and acoustic analysis and acoustic optimization all these problems fall into a broad class of techniques which are applied to problem solving applications and in order to take advantage of computers for problem solving designers need to formalize the design problem in terms of design objectives and variables but there is a stage preceding problem solving which i define problem learning and this stage is what allows the science to define and elaborate a design idea probably learning is a stage which allows the scientists to start from an ill-defined goal and turn it into a framing strategy and finally a design idea through the acquisition of additional knowledge about the problem so if for problem solving we use a modus of design mechanism for problem learning we need models of the design cognition so what i propose in my phd is to develop new computational design techniques which allows the designer to engage in in the creative exploration and new design possibilities and i'm gonna do this through artificial neural networks so i've identified the three different uh problem learning mechanisms these are expertise playfulness and analogy construction expertise involves the clustering and retrieving of information related to a single knowledge domain relationship is interacting with objects we know but under is and analogy construction is transferring knowledge among multiple knowledge domains so i'm going to show you now some applications which demonstrate how i've developed computational design tools to account for these three different problem learning mechanisms in the first application i've developed a data set of four t-shirts and special structures and trained operational encoder to extract design features from this data set to construct a design space the designer can then interact with this design space and start interpolating within existing design or also draw a sketch and ask the model to interpret it and generate the design proposition i've implemented the training model in a cad interface which allows the designer to sketch on a conventional cad software and get the results from the machine in in 3d so here we see the interface which consists in three nodes um we have uh the the cut software then um a cad scripting environment which allows to to the flow of information between the uh cut software and the machine learning platform here i tested the ability of the model to interpret existing designs and this represents three typical architectural problems like joining two different profiles in the first case in the second case an l-shaped footprint or joining two volumes together and we see how the model was able to interpret these footprints by proposing alternative ways of solving these conventional problems in the second application i try to understand what are the properties of design space constructed by an ai model so i've constructed a design space with the conventional approach by defining a parametric model and then use a samples from this design space to train a variation variational encoder to construct a new design space then i compare these two design spaces in a optimization procedure to test what are the solutions that can be retrieved from these two different design spaces and the objectives of the optimization process were structural analysis performed with the femme analysis and then two different architectural objectives to account for features like the opening site or the footprint shape and on the top you can see the solutions retrieved from the human defined design spaces which are extremely conservative we don't have much variation while on the bottom you see the solution retrieved from the ai generated design spaces and you can see that there are many variations and a lot of different topologies which could not be found in the human defined design space in this application i train the model to interpret an architectural sketch and to construct an image out of it i've implemented well-known pix2pix generative adversarial network and i've constructed a dataset specifically for this task then after training i've implemented it with a graphic interface and you can see here that the modeling interactively is able to interpret a very few strokes and return progressively a very detailed and inconsistent uh picture out of it so this allows the design to explore different possibilities but just by using a very abstract form of representation which is congenial to the exploration during the creative process so now we are moving through playfulness and the idea behind modeling playfulness is that a kid without any expertise knowledge is able to engage in creative exploration and it does that by intrinsic motivation so i try to model this approach of the kid to explore new possibilities um by via reinforcement learning but before moving to this complex task i've devised a benchmark um in which the the agent needs to construct a two d frames and on the left you see the environment with which the rl model interacts with there is a black canvas and then an obstacle and the model needs to construct 2d frame structures by moving the cursor in in the space and covers and when it completes the structure it gets feedback from a film analysis in terms of structural performance you can see here the stages of the exploration the agent is exposed to different obstacles and during the convergence of the training process you can see that it starts creating structures which are not just complete but also optimal and here you can see how the model can be used on the left you see that model is able to instantly optimize structures for a variety of boundary conditions what's most interesting on the right you can see that the model can interact with the designer so the gray part is the structure drawn by the design and when in interrupts you can ask the machine to complete the structure so a limit to this benchmark is that you can see that there is no much variation in and there is not creativity in the solution generated by the the model one of the reasons that the task is really naive um that is there is not enough room for the model to try to explore different solutions because the for these sites of the of the canvas solution is pretty straightforward but another problem is that uh we are not modeling a correctly the the process of playfulness in which kids engage um for example here on the left you see an image of a structure that is generated by is produced by a kid not just by placing blocks randomly but it follows patterns and where these patterns come from so freuble understood that there is some kind of patterns that the kids are aware of and these patterns come from the observation of nature so you you see the repetition of color pattern and also the use of symmetry so we are moving now towards the third application which is analogical reasoning um what i'm going to do next in my phd is to force these models to use patterns from a data set of natural images or fractal data sets and to reduce them in the generation of a design proposition and this is a matter of analogical reasoning because we are moving from one domain which is nature to another one which is that built environment so in the hand what i'm i'm trying to do is to the moving the first steps toward the development of the ai based cad system which are not focused on problem solving but on a different stage of the design process which is specifically focused on idea generation so what i think is that um ai should be used to support and enhance the the creative potential of the of humans and not to replace them so if you want to know more about my research please visit my website or contact me if you want to get in touch for future collaboration thank you thank you that was beautiful um thank you so uh we are handing it over to alexandra sorry i was muted let me share my screen and then oh sorry there we go uh does that does ever can everyone see my slides yes perfectly all good awesome awesome thank you um so my name is alexandra carlson and i'm a phd student in the robotics and computer vision departments at the university of michigan ann arbor and i've been working with matthias del campo for the last several years and we've actually kind of through our work have formed this lab called architecture and artificial intelligence the art architecture artificial intelligence lab at the university of michigan um and so today my presentation uh is titled defining and really exploring artificial creative processes um there we go um so the two points um that i'm going to kind of be focusing on today are um one how do ais contribute uh to creative and novel design and i've sort of subtitled this as human in the loop ai creativity and so in this uh initially i would kind of like to discuss sort of the pros and cons and sort of abilities of what these perception and vision algorithms actually can do and then finally i'm going to be presenting some recent work that we've done about encoding abstracted concepts in 3d using neural networks so the problem of creativity is typically always been considered a human one and the question of whether an algorithm possesses this ability is you know hotly contested um neural networks themselves though can uniquely contribute to uh the discussion um and development of creative design methods um these generative ai systems are able to ingest and learn from huge corpuses of images that can span cultural and historical dimensions that a human or humans would take a lifetime to synthesize and learn this means that they are potentially capable of generating solutions that could combine either unique patterns within a particular cultural historical context that a human might not detect or that they would be able to create plan solutions that combine patterns from different contexts without the influence of uh human bias uh furthermore uh the efficacy of neural networks to emulate and surpass the performance of biological vision systems on certain tasks make them a particularly unique computational design tool for generating novel outputs for example neural network architectures can achieve higher than human performance on image classification tasks suggesting that these algorithms extract visual representations that are better for classification in comparison to the ones extracted by their human counterparts um the learned visual representations of neural networks could capture new ways of seeing and understanding aesthetics with this in mind we can use these learned feature sets from the networks to achieve novel representations of form for standard architectural motifs um but you know in this context novelty isn't only the determining factor when it comes to creativity um and so kind of continuing in the line with bowdoin's thought process when we think of defining creativity itself um there's three things that kind of go into evaluating whether or not a process is creative and the first type uh involves uh novel or improbable combinations of familiar ideas um or combinatorial creativity and um so if we think about kind of like different ideas and ways to combine them we can kind of think of feature spaces where there's different dimensions that define our conceptual space and we know that these features manifest in different ways uh in the natural world in in our reality and um ideally what we want then when we talk about combinatorial creativity is to find um find combinations of these features that don't follow the natural distribution of them um and a great example of this is style transfer where our features in this sense are different textual elements that we can extract from the features and neural networks themselves can extract very different and geometric different features in terms of geometry and style and transfer them in different ways and so we've uh at the artificial architecture artificial and architectural intelligence laboratory have used this to generate novel city maps we've used this to create 3d textual maps in designing garden elements like boulders and um yeah so the set this brings us to the second we're exploring the potential of conceptual spaces and closely related third is making transformations that enable uh the generation of previously impossible thoughts um and i talked about these two together because they're they're somewhat interrelated in a sense that they deal with taking these feature dimensions and tweaking them um or drastically altering them and i think the neural network algorithm that really lends itself to these uh two types of or i should say these two subsets of creativity are gans gans themselves are able to learn features from the data set without much specification and so they're able to learn these distributions and interpolate between them very well and so we in in the lab have used these to generate um novel plans that combine natural elements as well as traditional floor plan elements and we've also used them uh to combine different stylistic elements of uh different uh historical historical styles like baroque and modern the issue with ganso and this is really what kind of gets to the cons of using or i should say the limitations of using neural networks for creative design is that we can't really uh they don't really extrapolate well outside of their defined their defined dimensions and so this is kind of uh where we sort of hit the uh the limitations of neural networks and where i would argue that ai is still i would say considered to be an assistant or tool as opposed to a contributor themselves and i think you know irrespective of what type of neural network that we're using we as humans um and designers still need to define the conceptual spaces primarily we are the ones that generate the data sets um and we create these data sets in different ways um you know by choosing what visual information we want represented in them uh and then also too i think a really important point to make is that neural networks are static functions we train them on these datasets they learn to extract these different features that might allow for novel combinations um and allow for combinatorial and um exploratory creativity but then uh they don't generalize well which means that this uh you know transformal uh transformational creativity is sort of still the holy grail um for us and so um this is sort of defined uh these limitations kind of have sort of defined how we try to apply these static uh these static function approximators uh two new data sets and that's what brings me to encoding abstracted concepts in 3d um and so oh i see that i'm running out of time very quickly the general idea in this project is that we wanted to leverage a very specific type of neural network graph convolutional neural networks um to learn a mapping from 3d space to subsets of creativity and in this in this sense we chose sensibility we chose aesthetic functionality semantics and because these all serve as very strong priors for what we did define as creative processes and so we created a novel sensibility data set where each of these models they're split between houses and columns they varied along style axes they varied my slides i apologize they're not enhancing they varied along functionality axes and then they also varied along aesthetic aesthetic axes and so we then trained a graft neural network um to essentially essentially learn sorry my slides are going crazy sorry uh give me one second um uh well sorry i'm having some technical oh there we go so we essentially trained a graph neural network to uh learn a mapping between um these ah i have to apologize um my slides are advancing and back advancing in an unexpected and unanticipated way [Music] give me one second i guess i can as my slides are kind of going insane uh we essentially generated a created graph neural network to learn to see if we could learn a functional mapping that approximates the sensibility process of a designer and we then uh in a similar vein to deep dream inverted this uh inverted this neural network to try to transfer the learned sensibility features uh to different simplified uh mesh models and the results now that my computer hopefully has stopped freaking out uh um these results are shown here where we are able to i have to apologize this is very frustrating um uh the results were hopefully going to be shown right now there we go um where we're able to actually uh simplified input models like cubes cylinders and octahedrons into interesting semantic shapes that have different aesthetic varying aesthetic and functional qualities um and we see that um as you know even though these feature elements are are fairly nuanced um we do see that the neural network is learning to associate specific mesh features um with specific uh aesthetic and functional uh functional elements and something that we went through and did is a way to kind of visualize uh specific features uh within the messages that are contributing to these evaluations is we use something called gradient-based visual saliency which is uh similar to deep dream where we invert the network and look at what features of the mesh are actually contributing to a high aesthetic rating or a high functionality rating and we see that you know the story is not super clear because in certain meshes um in certain mesh features uh you know we will see like at the bottom that um certain uh certain vertices actually contribute similarly to aesthetics and style but in other meshes that is not the case so what we have also gone through and done as a way to kind of further understand how mesh quality and vertex relationships can contribute to uh aesthetic and functionality rating um is uh we kind of we we leveraged some of these really nice um statistical uh regression analyses and have demonstrated that things like mesh volume discrete gaussian curvature vertices um as well as average face area actually contribute to uh they're directly correlated with aesthetics and functionality um which is which is very interesting um and so you know we have through our work uh demonstrated that you know this idea of uh subjection like this idea of subjective beauty um which is really important in driving creativity um can be achieved uh but uh that you know kind of the future work is going to be looking at how we can kind of escape sort of the limitations of um fixed data sets and see if that neural networks can actually learn from scratch uh thank you thank you so much that was really really interesting um and never mind we we understood everything even though the slides were kind of jumping around you did a great job thank you i again i apologize okay good i i have to really apologize about the slides um i'm not i think my computer is just overwhelmed i apologize no that's all good thank you um and now we're moving we're moving on uh to the next presenter uh which will be sorry i have to check my notes real quick um so next we have uh x cool presenting way long you're already on and you have chiang jiamin okay okay just a second no problem oh oh i okay i have to share the stream but i have to leave meeting first sorry no problem but i'm sure we have a lot of questions for alexandra so um this is uh goes out to the audience and also you know the other presenters um please type the questions in the chat on youtube and we will address them to alexandra and yeah other questions from from the other presenters maybe also take a note and we'll ask them during the discussion after the presenters so thank you can i just leave a thought up there while waiting for the presentation yes sure should we still be using the term artificial intelligence um that's the main thing i want to ask whether we can think of either a different term like intelligence or another term like synthetic intelligence i just will leave that with you um as a concept because i think because the term artificial is not terribly useful in many ways um i accept completely alexander's arguments about uh about the distinction between humans and and machines but i want to throw that out there whether the terminology is not very useful yeah i think uh that's a very interesting discussion and i think we should definitely talk about that as a group after because uh yeah we see we see the limitations but we also see the um opportunities that that are coming out of this and there's there's a lot going on and uh limiting ourselves with terminology can be actually very not only challenging but maybe even dangerous so i want to continue with that discussion definitely so thank you neil okay i came back so were you able to figure out otherwise we will continue with um emmanuel okay that that worked perfect okay okay awesome stage is yours okay thank you it's all nice to be here i'm and my name is leo weilong i'm a software engineer from xcodetech today i'm very happy to share with you the current research progress made by our architecture research team in xcodetech and today our topic mainly focuses on how we use ai to create representation of architecture design especially to create mass plan and site plan so this quick talk is consist of four parts first i'll go through the background of this topic then i will talk about the problems we face and our solution lastly i will give a short introduction about how we use ai to create flight plans so let's start with the first part and architecture designs continuous process of creation is heavily based on visual language so architects have to express their ideas and proposals of each design stage by images or drawings since the design itself is so complicated that we cannot explain them only by words human beings has been learning how to use drawings and images to represent architecture design since long time ago and as building industrial growth some joints have become parts of the standard delivery such as the floor plan the master plan flight plan section plan and so on only with these standard drawings can we efficiently convey our ideas to others so cycling is one of the standard drawing used by architects urban planner and engineers it shows the condition of a given block as well as the possibilities of arrangement of buildings parking lots paths landscapes and it also can be used as base of site analysis transportation and urban planning use a site plan is convenient to convey ideas and concepts because it reflects concept and design intent over architects so as a result architects would possibly take a long time on drawing perfect site plan in order to represent their design properly so a site plan is important but it's embarrassing that normally we have very limited time for joining a site plan typically when joining a site plan the architect has to clean up lights from sadie or rhino choose texture and style and plan to plan the landscape and the difficult part is we have to do the whole process all over again if we change the original design and all this tedious work must be done within this short period of time and it's not it's a work it's not a easy work even for senior architects so here we have two problems one for quick landscape planning and the other for dealing with repeating manual rendering so the first so first problem is plan the landscape as quick as quickly as possible so we try to solve this problem by using ai and and last year we participate in digital future workshop our topic was about master sketch we mainly focus on um letting the ai model learn how architects turn their sketch into a valid net image just like the way they did in conception design stage user can draw whatever they want as input and our ai model is responsible for translating the sketch into a image with some creative landscape generation um yeah this is uh so our amole is based on gang namely the generative adversary network um and the main idea of gang is that two neural networks contact with each other and during the training the generator network tries to create the cycle picture as realistic as possible while the discriminator tries to attack a picture fake or not for a set of image which contain both fake and real images and the game result will feedback to both discriminator and generator and finally the generator will um converge and then the output of generator should be in this screen indistinguishable from the rear sight feature and the the concept is easy to understand but to build a system like that it still cost us a lot of work as you can see the output of our okay the output of our ai model needs further translation so we have tried several mechanisms to make the output useful and here is the output our first trial is to design an algorithm for transforming to the output into 3d models as you can see in the video we might gain a better sense of scaling from the 3d model which means a lot of parking which means a lot for architects and well okay okay and the second trial is to improve the quality of the output the intuitive way to do so is to feed more data to our model and that's thus thus we collect thousands of new psyton pictures cut them into pieces due to the same scaling and feed them to our model it helps improve the texture and the color style of the output and it's the part of the training data set so after the workshop we have uh pushed our research further trying to explore the limit of what gam model can do on generating landscape by giving owning a draft so we redesign our model and increase the number of parameters inside by nearly one order of magnitude and then we fit it with small data and then as you can see our ai model began to get more sense about how to generate trees and buildings based on simple hand draft and if we increase the training iteration the boundary of the landscape and buildings will become more clear so due to the previous research we have learned the basic idea of using ai in generating mast plan and site plan and the next step is to build an application for generating insight and picture so our research and engineering design uh developed a very complex system to teach our model uh to create fun picture so our system contains three different deep neural network for three different functionalities uh the first neural network maps the input data which in most case the cid files and read the files and map the data inside into um latin space and then feed the data into the second neural network the second neural network generates the landscape while keep the buildings and block boundary unchanged and the third new neural network is the set of uh is a set of style transformers which can transform the original drawing into different styles and finally we send the output of network to our render engine and get final image in several formats so there are several advantages uh using our apps for site plan generation the first was we reduced the time cost from three hours average to uh nearly five minutes and you don't have to worry about changing the original design and in the last 10 minutes secondly we reduced the possibility of software crashing especially for those who have to switch between cad photoshop illustrator and so on and lastly we our app can be used as a inspiration trigger when architects have no idea about how to do their design they can use the app and here we have our demo it has here's the demo okay here's the um 2d 3d input and the 2d output and that's it that's all for today thank you for listening thank you thank you thank you very much that was uh great with the subtitles as well so thank you so i'm pretty sure we have a lot of questions about all of this um so again please if you have questions and you're watching this on youtube please put the questions in the chat so we can address the speakers after so thank you um and we are moving on to emmanuel ramiso from florida atlantic university stage is yours uh we're alone would you mind unsharing your screen so we can share stream okay thank you good afternoon everybody so um it's a pleasure to be here and thank you for the invitation um my presentation is going to be a little bit less practical than the um the ones we've we've seen until now um my personal interest is the relationship between uh human cognition and and machine cognition and so uh currently i'm i'm an associate professor at florida atlantic university and i'm also a doctoral candidate at the university of patras in greece where i'm um looking at this sort of inquiry which which can can help us kind of evaluate human creativity through the filter of computational creativity and vice versa of course right so um you know in light of the very short time frame i will probably have to skip some information but the main idea here is to talk about abstraction abstract thought as a kind of factor for cognitive development in humans and how that it becomes important for um establishing conceptual domains which can help extrapolate the way we pursue computational creativity so in this in this role we can talk about two aspects which are reproductive and creative imagination right so creativity is clearly a very complex um a very rich concept because it's often embedded in other concepts it has traditionally been the object of inquiry of both philosophers and scientists alike um can't believe that genius requires both imagination and understanding but stated that the relationship between the two is kind of fluctuating so they can they can serve as a kind of primary or secondary roles depending on whether these are used for cognition or aesthetic purposes um and i think while this is not entirely accurate it echoes this kind of certain duality which sets apart strict purpose-driven tasks for more open-ended ones and this is of course further more indicative of the differentiated subroutes between regions of the left and right hemisphere of the brain the relationship between the parts to the whole convergence versus divergent thinking which is necessary for intelligence and and so on um and so um probably all of the above are antithetical but also complementary uh relationships so as i already mentioned creativity is kind of a richer concept on intelligence because intelligence is a necessary but not sufficient condition for for creativity meaning that a certain threshold of of intelligence which typically is measured to an iq of 120 is necessary but beyond that the the the correlation is not is not linear because there are other factors which become more dominant in terms of defining creative output um one one of these packers may be contextual uh uh since um this is an interesting example by uh a comment by a doctor john sulk who actually discovered the the polio vaccine and his his particular feeling was that uh at least he's his own awareness in his own awareness uh he he was inspired to to actually arrive to the solution to that particular uh task by uh while being in the in this in the town of assisi in italy um and of course these this can open up a kind of other kind of uh discussion about the role of conscious versus unconscious role uh of of um of thinking within uh establishing a solution for a given problem which which of course goes back to poincare's distinction between um conscious unconscious uh thought illumination and verification right and what's what's um interesting to notice that the the architecture of the salt institute which was later built in california is very similar into topographically to the town of assisi um so well several what i want to do is briefly talk about some kind of the earlier foundational work which was done about cognitive development almost 100 years ago by um by a scientist called alexander luria and um this kind of stresses the importance of language as a vessel which allows a transition from our sensory to our rational consciousness right although although several other scientists like einstein galton and hadamard mentioned that verbalization is not always necessary for thinking i want to stay in the importance of language at least for for the sake of this particular discussion so around the the night the end of the 19th century um it was wrongly assumed that all logical categories which allow complex thinking like inference and imagination exist by default in the child's brain from from birth but according to professor luria it was a sweet psychologist john piaget who first showed that the basic processes of logical thought in the form of induction and deduction are a result of development and that in the earlier stages of children's cognitive activity those logical processes are replaced by less sophisticated forms of transduction in which direct impressions play a much greater part than the as yet underdeveloped verbal and logical schemas here's just work later on to the led to the kind of development of the field of genetic logic which regards logical categories as a result of a complex or a more complex psychological development so luria went on to conduct a series of extensive surveys in rural parts of the former ussr during the end of the 1930s in order to document complete development conditioned developmental differences between different social groups who had little or no formal education um so in in this in a couple of next slides you can see examples transcripts from from the interviews which basically demonstrate that uh essentially the group among which were primarily literal illiterate uh people from remote villages had a very hard time reasoning based on anything anything other than their own practical experience so he's luria conducted several experiments but the two i mentioned here are the one about deduction inferenced which was based on syllogisms which is what and you can see some of the examples of questioning uh here so um for example asking someone whether giving someone a statement about two particular facts like cotton can grow only where it's hot and dry in england it's cold and damp can cause and grow there i don't know think about it and and the subject response i've only been in the kashgar country i don't know beyond that um so i think this is kind of um on one hand fairly obvious today but on you know i think on kind of within the context of uh uh the early 20th century this is one of the important kind of foundational works that begins to kind of infer the distinctions between reconstructive imagination which essentially depends on on memory and practical experience already lived and then creative imagination which allows more abstract reasoning to lead to inference and hypothetical situations in this page we see this kind of next line of experiments which was more targeting imagination through free questioning and for example the subjects were asked to propose any kind of question to the interviewer in terms of learning about new things and it would be uh they were very kind of not reluctant because they were quite eager to actually engage in anything that was relevant to something that they had already experienced but it was very difficult to actually imagine even what to ask so if you look at the second transcript from from from the right you can see the subject is asked what would you like to see other countries other cities and what would you like to learn about them and his response is that probably there are interesting cities as you say but i don't know what's interesting about them they took my horse away and the road is long i can't even imagine how it would get there so um i think it's kind of important to understand that while an interest is revealed in their own direct experience they have a difficult difficulty going beyond beyond that um so i think ultimately all the results demonstrate that um the social organization and education facilitated the removal of cognition from the realm of the concrete to one where hypothetical logistic thinking can take place and thereby it highlights the difference between reconstructive imagination which allows mental processing based on prior experience through access to memory and the more complex creative imagination which deepened on abstract reasoning depending on abstract reasoning it is clear that the combination of both types of imagination is necessary for creative thinking to to occur if we take a step back just to kind of uh look at from an evolutionary perspective you know our the human brain expanded over time and what happened is that the the distance between the centers between the areas where you would have where you have a certain input and a certain output grew larger and i think that created a richer possibility for for creating new associations um furthermore you know the that expansion specifically allowed of course the frontal cortex to to get larger which is considered to be one of the primary uh regions of the brain for um for imagination from an anatomical standpoint it's important to understand the differentiation in in the hemispheres or in the symmetry of the brain because um the left hemisphere although it's widely acknowledged that both hemispheres regions from the both hemispheres are simultaneously activated during any kind of mental task uh it is still um uh accepted that you know that the left hand is here or the or parts of the of the left hemisphere are responsible for tasks which are driven by a particular purpose whereas uh the right hemisphere often just reaches out so to speak without any preconception and therefore entails a more heuristic nature of operation when it comes to the creative process this also highlights of course the qualitative relationship of the part to to the whole the relationship that relationship is important in guiding our understanding of artificial intelligence systems the reductionist view of ai is not very helpful you know in terms of for example you know understanding you know what um an artificial intelligence can can perform by by looking at the literal translation from the human uh set of conditions to the artificial set because um as we have learned from complexity sciences the complex systems cannot be understood by the study of the individual parts because it's a high level of performance which results out of the interaction of of these subs so these are these parts this kind of this kind of concept can be understood if we look at holistic or also known as gestalt perception the right hemispheres is the whole before whatever it is gets broken up into parts in our attempt to know it it's holistic processing of visual form is not based on summation of parts on the other hand the left hemisphere sees part objects the process is not a gradual putting together of bits of information but in a half phenomenon which comes all at once the right hemisphere with its greater integrative power is constantly searching for patterns and things so in this um i think slide we can we can see that um that kind of differentiation between our reading our our intention our tendency to read both apart in the whole at the same time however uh the the you know the number four and the the letter h would be perceived before the number the letter e and the number eight in the in the smaller scale furthermore you know the um in the image of the right once suddenly you know after after a few seconds begins to kind of understand that there is this kind of uh mixing between the landscape and the dalmatian dog you know which are all belong you know kind of uh inferring this sort of similar pattern at the bottom we can see that the the right hemisphere clearly is able to recognize uh an incomplete part of the whole but having a more holistic understanding of the structure of particular mental image even when the left hemisphere is inhibited whereas on the left diagram here if the right hemisphere is inhibited and we leave the only activated activation of the left hemisphere we cannot perceive the complete picture but we only get a fragmentation of a mental image the non-direct correspondence between humans and machines lies at the heart of this paradox which was proposed by by hans morovic in the in the late 80s um more of which acknowledged that tasks which are easy for humans like sensory motor skills are the hardest for ai and vice versa marvin minsky also emphasizes that the skills which are the least conscious of are the hardest to reverse engineer and i think this comment is indicative of the complex nature of cognitive processes in the brain where various brain regions complement each other mathematician roger penrose provides some insight on the role of the cerebral and cerebellar regions during the execution of a simple task once the task has been learned using centers in the cerebral region it is passed on to the cerebellar region where where the task is performed automatically so to speak returning to the analytical process of the learning by trying to describe the steps involved in the learning can completely obstruct one's performance as it kind of forces us to rethink something which has already been adopted albeit on an unconscious level and so what's interesting is that um in particular tasks where where consciousness or where where the time reflects is based on fragmentation of the overall process like activities like tennis or even ping-pong which is actually requires much much faster reflexes um the human response would be guided by this kind of more automatic so to speak mode where uh the brain is under cerebellar control because it has already mastered uh the skill of returning um a particular uh a particular hit by the opponent and thereby uh it's almost operating on an unconscious level this is interesting with regards to um i was thinking about the the promotional video which came out a few years ago and i'm sure some of you have seen it from kuka robotics which was trying to kind of emphasize on the on the speed and flexibility of the agilis robot in this kind of hypothetical duel between the robot and the former world champion world champion in ping pong team apollo um so what's interesting is to going back to the comment about the cerebellar control to understand that perhaps who's more conscious in this kind of situation is it is it the robot or is it the uh the human who is almost on autopilot at this point according so i'm going going to moving on to this this kind of broader field of computational creativity according to deep mind co-founder demis hassabis um and i think some other people mentioned this today so i don't really want to kind of um repeat myself but just basically suffice to mention that we have those kind of three ways of understanding creativity from within an artificial intelligence context which is interpolation extrapolation and invention interpolation is what typically neural networks are very good at while extrapolation uh is some is an area where we are improving and of course that we are we're still quite quite far from from invention right so within within the work uh that if a you uh we looked at um the importance of domains of conceptual domains in terms of creating these kind of connectivities between conceptual spaces right and so we used in this project we used music as a kind of filter um to where where to understand how we can retrieve um let's say three-dimensional structures which can relate back to some kind of uh spatial conditions so we're using the musical structure as a generative tool to conceptualize space making frequency intensity and time were the parameters that were used to evaluate the structure of the sound then we used firefly to create the visualization the 3d point coordinates were used as an input for all to perform machine learning with a tsne algorithm the algorithm learned the structure of the music based on a particular genre representation and then transfer this to another musical piece and allow us to visualize it um this is still in progress but the intent is to exit the architectural domain and conceptualize space making in a more flexible manner looking at qualities which would otherwise be not addressed in hindsight of course because the selection of the particular musical genre here is is specifically important we understand that the knowledge of that particular conceptual domain is important if we want to reach more tangible results and i think i'm going to get back to the domain um discussion in a minute because i think basically according to um to current research results the familiarity with a particular conceptual domain is important because it both allows the designer to engage in some pre-automatic process which has already been learned without necessarily having to to go through an entire search process but at the same time it increases the chances for persistence um we in the sense that breaking a particular conceptual buyer that may be necessary to to reach to make that creative leap um and so this is another project which daniel bolojan and and and i uh began in 2019 as part of a workshop taught at the ikadi um conference where we're looking at domain transfer using cycle guns and and primarily um extrapolating from accelerating relationships or interpolating relationships between the sagrada familia and different um and other conceptual structures like mathematical deformation or interpolation between one particular style and a different particular style but i think the most important uh category was that which ex exited the um let's say the known or the typical customary relationships and looked at the the notion of a forest and the interpolation between the sagrada familia and and uh in the forest so what's important here is that as our own kind of self-critique the um and kind of situating within a broader discussion about about domains is that the let's say the um the qualitative nature of the domain is still decided by by the agent and not um not offered by by by the network and so this i think still remains an important barrier in how we kind of engage with or open up the possibility for a computational creativity that goes beyond interpolation going back to some of the challenges in relation to this kind of conceptual domain and broader understanding of data within within ai i just wanted to kind of uh you know show a few examples from from art which i feel are indicative of the differentiation between and the illustration of the of the extreme complexity of the of the way that the human mind works in relation to how the neural network understands data you know in kind of situations where you have this kind of recursive structures like in in escher's drawings um human the human brain automatically can infer the different relationship of different or the same uh particular pattern in different scales but a neural network perhaps may not be able to do so at least in a straightforward manner or in timely fashion unless it has this kind of labeled uh more supervised it has to go through a more supervised process to understand that beyond a certain moment of abstraction a particular geometry is still inferring the the conceptual association to that original object which for example in the in the last exam image who sees is a butterfly right at which point does the butterfly stop being a butterfly and becomes this kind of insignificant geometry so um i want to end with a few comments on on broader human creativity with regards to a process that we were we put together uh between um uh my colleagues daniel baldwin and sherman yosef and and which was partly implemented in the digital futures workshop last summer what we tried to do is essentially to to establish an understanding of assessing the computational aspect of creativity not so much by looking at the results alone but looking at the the structure of the process and i think this is important in relation to my earlier comment about domain association because that allows this kind of flexibility in terms of connecting and let's say chaining input and output from one particular neural network which of course is inherently um assisted by a particular human designer and another neural network which which could be assisted by different human designer so even though within this kind of scope of work the the supervision and curating of the data sets was was was happened with the guidance of you know the three instructors we would like to to to propose a possibility where where the conceptual understanding the conceptual differentiation of of the different tasks as they pertinent to each particular group of of neural networks is um is contingent and and um let's say enhanced by understanding and retaining this independence among the groups so so with regards to um to that relationship between ai and humans i think it clearly illustrates a necessity for human and machine collaboration which can lead this kind of what we would like to call like monthly design creativity uh where in in this kind of operative framework the process would become of primary importance this has been also demonstrated by a lot of other theories including former world church champion gary kasparov who um who studied um various combinations of human and ai players of various skills supported by weaker and stronger processes conclusion being that a strong process ultimately becomes more important than a strong player whether that is human or artificial i know that i'm probably over time so i'm going to stop and we can address any additional comments in the discussion session so thank you very much thank you that was really fascinating and uh i think it's it's really great to get that kind of context because i think it's interesting that you're mentioning um kind of the difference between the process and um the output or the you know the outcome and the product so i think it's it's it's very um it's very interesting and very um important in this discussion so i hope we can we can come back to that so um i i was reminded of you know francis charlay uh the measure of intelligence his famous paper and the interesting you know definition of what is intelligence i mean this is very important in this discussion and i think um a lot of us as designers kind of forget about the very big underlying consequences of all that that we actually still trying to discover what is actually intelligence so um thank you very much that was a really really great um contribution and i hope we can engage and contextualize a little bit more uh in the discussion so uh thank you we're moving on to the next um presentation um by the uh by martin martin sorry um okay would you like to share your work with us yes the stage is yours thank you very much hello everyone good morning my name is martin calvino i'm speaking to you from new jersey united states it's about 10 am on saturday just to um place you a little bit uh my talk will be non-conventional in the sense that i have been exploring of ways in in which i can communicate myself without the use of powerpoint or keynote software in ways that that my narrative is non-linear so i won't be describing work slide by slide it will be more interactive so thank you very much um to the organizers of the digital future talk series i'm very happy to be here sharing my work to with you so basically i am a multimedia artist which means i use several media to express myself a long research interest that i have basically i am interested in abstract painting the intersection of media arts with science in particular plant genomics machine learning and tango culture so this is my website so you can visit and this is my contact info if anyone is interested in contacting me and i also like to collaborate a lot with people from academy so if you find my work interesting so please don't hesitate to write me a few lines so today i will be discussing with you some of the work i've done with machine learning in particularly the use of deep convolutional generative adversarial networks for the generation of abstract paintings and but before that i just will briefly describe what i have done in in in machine learning so um at the beginning i have used recurrent neural networks for the generation of tango lyrics i am very much interested in in tango culture so what i have been doing is training training networks with about 6 000 tangolydics to create new new lyrics which contains the grammar from from from the past of my home country uruguay and argentina and create electronic music in which i i read aloud the generated lyrics from the algorithm and create a electronic music with with tango ranks i have also used um try current node and network to to to create a synthetic sequences that resemble those found in nature and and this work has the potential to create transgenic plants with machine-made synthetic gene copies which is very important because usually you cannot patent action that you can that you find in nature but you could intellectually protect a machine sequence that has been artificially constructed so my my first explorations with with art in this visual art and artificial intelligence has to do with image classification so basically what what i did was i was interested in in in in classifying using on classification algorithm to see if a convolutional neural network could could distinguish artworks that i did by hand by painting and drawing versus artworks that i did by writing computer code so which element would the machine recognize in my image as being distinctive from from from art i do by hand versus art that i do by computer code and the important one of the important things in this work is that i i use data sets entirely from my portfolio i don't assemble data sets from other artists image work or or pre-assembled data sets i create my own data sets from from my own paintings and drawing so basically this is an example of a data set of hand drawings made on acrylic on canvas or or digital drawings on on on a wacom tablet and these are examples of a dataset image dataset with with arts created with computer code usually the processing language sometimes python and sometimes um max msp jitter so basically i i i created an email data sets with images that are squared about 256 pixels by 256 and uh implemented a an image classification algorithm and i could effectively train the algorithm to recognize um images that derive from artworks made by hand versus adwords that were made with computer code so you can see the the classification accuracy is very high so here i give you 10 examples these are images that that the classification algorithm correctly classified as being drawn by hand only one error classified as being made with computer code with this is this image and the algorithm correctly classified all the examples that were created with computer code so the next question that the the next interesting question to address was okay which are the which parts of the image are being recognized by the algorithm that could distinguish uh handmade versus computer-made artwork so and i did address that on on the following work in which i dive right into um visualizing how a convolutional neural network recognizing visualizing active activation functions on on a convolutional neural network so what i did is try to to understand which which segment of the image the algorithm is recognizing as being to one category or the other so after i have implemented this algorithm i want to show you some interesting reasons so these are um activation maps that tells you which region of the image is being activated by by a feature or feature maps so you can see that on on the left you have images the eye from artworks created by hand and on the right in in a heat map like a red orange yellowish coloring is telling which part of the image the algorithm is is recognizing as important for classifica for classification of this image as as being done by hand and and this is the the the activation map so it's interesting that the algorithm is recognizing that the places in the image where i just stopped the line and kind of make a point when when i draw in and start the new line as being um as being a characteristic of handmade drawings and these are the activation maps for the images created with computer code but most of them the algorithm recognized them at the edge of the image you can see here the heat maps [Music] and and here so this is very clear in this image here in this very sharp um very sharp end of the this visual element compared to the background so just to give you an example so this image data set that i created contains about 6000 images and based on that image data said the next step was okay can i generate new abstract compositions based on this image so what i did was i use a dc dc gun algorithm from deep convolution and generative adversarial network with the same image data set i combined all the drawings made by hands with all the growings all the drawings paid but with computer code and i train a dc algorithm that you are familiar with with right to two networks a generator and a discriminator and these are uh the results so i did two runs two training runs of the dc gun algorithm the first was a test case with a small data sets or data set of about 740 images and these were the the preliminaries preliminary results i got as you can see very interesting visual compositions generated by the algorithm that somehow resembles the way i i do curves in my work you can see for instance this type of curves and and the color combinations uh were very surprising between what i do but at hand and what i do with computer code so the algorithm somehow helped me to visualize the color combinations that somehow were unconscious in my art practice but now the algorithm exposes to me in a very clear way that i can take on this combination of colors and textures and include them back into my artistic practice so what i did next was exactly that so i was inspired by the the artworks created by the algorithm and then i try to incorporate them back into my traditional abstract painting so you be inspired by color arrangement the disposition of visual elements within the image and then i will start creating my my own um series of abstract drawing completely inspired on on the images generated by the algorithm but combining them in my own artistic way without losing my own artistic identity i tried to avoid just copy the design of the generated by the algorithms and these are the results these are the drawings i have been doing so far i started last year and it's a continuing ongoing project so you can see these two examples in in black white and green we are inspired by by this r word generated by the algorithm so i borrow from this composition the colors that were used and try to combine them with my own drawings and make um a new artwork so these these images that you he you see here were generated by the dc gun with a bigger image data set of about uh 3000 images and you can see that as i increase the number of images on the training set the artworks become much more interesting visually much more appealing so what i did next so this re this process of training this process of creating my work training the algorithm to learn from my own work and then incorporating what the algorithm generates back into my my artistic practice re reminded me of of a concept that i have studied during my year as scientists in evolutionary ecology that we call co-evolution revolution is a concept that refers when when an evolutionary change in one species is goes along with an evolutionary change on a related species that usually interact so let's think about a prey and a predator when there is an evolutionary change on a prey on a prey that that increases his survival rate its survival rate relative to the predator the predator would also have an evolutionary change that will make him more effective in hunting the prey so in this sense is what i'm doing here so um a change in the in the algorithm that creates a new painting incorporate into my artistic practice change my art train again the algorithm and and keep and keep going so i named this process artistic process coevolutionism and this is the the overall process by which i do it so i have my artistic portfolio create my image that i said as input to the algorithm the out the algorithm with create will generate it's it's images that i call artificially in ai inspired art and then these images i will incorporate them back into my artistic portfolio so this is what so far what i've been doing is a continuous um continuous ongoing project just let me show you the new drawings that i have done so i have now a collection of about 30 drawings that are all inspired on on visual elements from from the generated images from the dc gun algorithm so these are some of the latest visual work i've been doing so in this case some these strong um yellow lines were inspired on drawings um by the artificial intelligence algorithm so just let me close it here finally just to to close i would like to share with you the next once i create once the machine generates this new abstract art i've been trying to to think about ways to take it out from the computer screen into the real world and one way i did that was by combining generators by a dc gun with fashion so this is the my first um result along this artistic line so this is a trouser where is made from from cotton twill fabric printed with one of the artworks with a collection of artworks created by by the dc gun and this trouser has a it's a very traditional trouser from my home country used by people in the countryside which is not that um appealing so what i did is i i totally remade them for a contemporary and urban outlook by printing on the printing on the fabric the ai generated work so this is basically what i have been we need to do i'm happy to share with you some other works i've done along with machine learning involves the automated detection of political ideology from text i was very interested in addressing if you could use um unsupervised machine learning algorithms such as topic modeling and clustering to classify i mean to to to see if collections of of articles in newspaper that were published uh during election during a presidential election in my home country if i could use machine learning to to detect political leaning or political bias of a newspaper tower one's political party or the others just very briefly i will show you this work and then i will end my presentation here so in this work i analyze around 513 newspaper articles from five different newspapers that we are writing about two political candidates that were running for president in my home country which is uruguay and i combine this with with a battery of of natural language processing um procedures and and this is the the the application of machine learning in which i use um clustering algorithm to distinguish newspaper articles relative to the documents of governance for each political party and then i extracted the topics for from these articles and then combine them with with with network graph analysis to see if if each use newspaper would cluster together with the with the documents of governance for one political party or the other so i have shared it with you this file with all the links if you if you like to check them out just do it and don't hesitate to write me on a few lines if you have any questions thank you so much thank you martin that was very interesting um i think the first presentation that was really you know showing the hand in hand work that you do with the machine so very fascinating i'm pretty sure we have a lot of questions but because of time we want to move on uh to the next presentation um by the current the current team from stefan institute so provides and evie if you want to get started and item of course stage is yours hi everyone we're the current team i'm professor architect researcher based in hong kong currently teaching at the bartlett school of architecture ucl and i'm ellie jotiva i'm an intermediate artist and researcher working at the intersection of art and science and i work in between los angeles and sophia hi i'm jens a researcher and vfx artist from moscow russia i'm arjun kenoski a data analyst and machine learning engineer based in moscow so we all met in 2019 at strelka the new normal postgraduate research program where we started current a speculative project on the future of broadcasting cinema as well as its impacts on our ecologies both cultural and environmental so current emerges from the intersection of four contemporary trends that's live streaming culture volumetric cinema ai optimization and personal personalized attention economies the speculative film current which you're looking at right now is an experiential example of what this kind of new cinema might look and feel like within a few years based on the convergence of these four trends in the process of training our machines to see and comprehend current anchored its data feed from live stream for its real time and crowdsourcing qualities streaming data channels from multiple sources and perspectives provide current with a means to outsource imagination the question of how we can facilitate an artificial creative common intelligence is at the forefront this points to a new form of creativity where the authorship is a participatory one and contextualize the relationship between ai and creativity in the creative commons the word creativity has a root in latin with a christian implication of creation from nothing a genesis from a larger being it is not until the 19th century that the term embedded itself in poetry science and art not anymore a mere form of repetition and propagation but a creativity that reconciles with rules from constructing and deconstructing a creation from something today creativity has encountered its third archetypical turn in the face of ai which can take forms of rule-based and machine learning systems the former involves the design of models with sets of rules the latter achieved intelligence with machines that define their own rules based on available data transcending creativity from causation to correlations from small to big data alongside the film karen experimented firsthand with a range of rule-based and machine learning systems that are readily available to any individuals and developed a production pipeline that provides a means for individuals to collectively reconstruct navigate and understand event landscapes that are often hidden from us such as the handling of trash and changes in narc animal behaviors in a process of iterative feedback filling invoice between sensory data into an endless stream of history where design intuition and algorithmic generation come together as a larger whole this is how current define artificial creative common intelligence in current we're talking about a personalized volumetric livestream cinema which means you will navigate the 3d world composed of live streams and other types of media forming your environment users can use not only 3d generated data but all the previous heritage of video content could also be converted into this 3d space for example we use technologies like photogrammetry and monocular depth estimation for reconstruction of some environments in our film just from thousands of hours of youtube videos so if all this comes into one place you'll get a huge real-time simulation with its own architecture based on your preferences these preferences could be calculated in real time according to your mood like the new suggestion algorithm in development right now by spotify which makes this decision according to your voice and other inputs every slice of this simulation is a film on its own carefully arranged by suggestion algorithm with the ai hand-picked moments in time and space a seamless 3d fabric woven by ai the rise of convolutional neural networks made a revolution in computer vision and opens a way to very in intricate algorithms like object recognition image upscaling and image synthesis with generated adversarial networks of guns this made us think that computer finally got the ability to understand the world around us moreover computer became creative they have imagination with the help of ai we can generate new worlds never seen before by creating images or videos out of previously seen content adjusted to specific scenarios either generated or handwritten to satisfies the viewer's desires the development of autonomous cars acquired the change of paradigm of machine vision from 2d to 3d vision for better navigation in urban environment and for more adequate understanding of what is happening around us thanks to this we are now witnessing a new revolution of ai now it sinks in 3d this bring us better 3ds since reconstructions even from one shot faster and more realistic rendering but also new algorithms of semantic segmentation which brings the ability to recognize objects and their actions in volumetric space now guns can fantasize in 3d by generating not images but the objects themselves and our contemporary attention economy commodifies users attention for data licensing from business to business currently often leaving individuals users with little to no control over the value that their attention generates and where it goes so we're asking like what if we're able to interact peer-to-peer and collectively direct the value of our attention to volatile environments marked by instability and contested events both artificial and natural in this way current raises the question for socioeconomic inclusivity of non-human vantage points for example in the contemporary shift from centralized networking platforms to decentralization through crowdsourcing protocols or the design of rule sets that guide our data communications this inspired us to consider how streaming big data that reflects real-time climate change and multiple animal points of view can result in maybe landscapes that begin to own themselves we ask how can a decentralized network of humans machines and environments extend their network effects from the virtual back to the physical these are some of the core research questions behind currency which is our most recent iteration of current in it we propose a volumetric live stream platform that can harvest attention values and then return them to the endangered environments caused by rapid urbanization by placing these volumetric reconstructions of volatile ecological sites within virtual events we can foster more awareness attention and tangible support for them through distributed blockchain technologies within currency we also search for the role of ai in our attention economy we ask questions like how can machine learning assist us in directing values within distributed contexts and can we imagine a crowd-sourced coded intelligence that consistently updates itself as a universal protocol and perhaps more importantly how can these new synthesized networks between machine human and environment stimulate new imaginations and creative expressions artificial intelligence increasingly molds the clay of the cinematic image optimizing its vocabulary to project information in a more dynamic space embedding data in visuals and directing a new way of seeing from planet to global flat to volumetric personal interplanetary with these collectively reconstructed and synthesized worlds current will be hosting a series of virtual events to expand our modes of interaction in the volumetric space in 2021 including 3d chess play virtual signaling fashion show and swire a distance aware real-time audio data system a volumetric choir the consistent effort should not be credited to current alone but also our international network of talents current always welcome anyone who wishes to collaborate we have been very lucky in receiving many hands who have reached out to us in facilitating an artificial creative common intelligence for opportunities participating and collaborating in our upcoming virtual events please visit our website www.current.com and our instagram page at current.com thank you so much thank you thanks for watching thank you that's fantastic thank you guys so much i mean it's it's uh it's provocative but also fascinating i i really really enjoy it thank you and i was actually reminded of what james lovelock uh was describing in his book nova scene where basically the um the agency will be given to machines to take control over um planet and and uh weather conditions because we as humans are failing at it um due to our chaos so i thought it was really interesting that you guys are incorporating uh thoughts like that um which are very provocative so um i would love to hear other people's comments about this but before we do so um please michaela take the stage as our last presenter today fascinating topic and i hope we continue the discussion after right thank you i loved uh i loved the the current presentation so i'm happy that i'm presenting after them but also that's kind of a hard job because that the presentation was great um so i'll be talking about do you see now my screen my presentation just to yeah yeah yes okay great um so my computer is ah sorry so i will be first i wanted to introduce myself my name is michael pinachakova and and this is also my presentation is going berserk a little bit so i'm going to wait at the moment stop share and uh if you think it would remove the windows from the zoom there's certain areas a little bit covered from the zoom window do you mean which it's probably the chat or the participants that are covering it great okay so i have to so i'll do the speaker view i guess and then you'll see only okay let's try again now i'll put is this better it's taking a moment i can't see it yet i think you need to un unshare and then then select the particular icon on your screen that you want to share maybe try it again unshare and share again okay i'll do that sorry okay oh okay so share okay now it won't find the window that i wanted it to so that's great okay i think i closed the window that i this is embarrassing um but anyway so i'll i'll start talking about myself so i am an xr creator and a phd student at york university in toronto canada originally i come from slovakia and i used to be so i'm not in computational arts i am i am in actually cinema media studies so i will be talking about ai and human in co-creation as a form of storytelling today um and i'm trying to find the window that i closed but i will um this is like stress and um okay here we go no worries i mean uh while you do that i just wanted to say that uh please everyone get your questions ready and i think we're we're excited to have the discussion um i think that it's been a wonderful morning at least from california standards and i think we're almost ready to go um i would actually um invite oh i think we're we're there thank you yeah we can see it now i'm going to awesome you can see it now i'm trying to do the presentation mode i think i had the same same thing with the google slides as uh one of our speakers had but i also did i i have it also in uh pdf so i might i might do that yeah um it says present from beginning so maybe you just okay let's go it's going perfect there we go okay so okay but i need my percent of you god this is the zoom is making me crazy anyway okay so i'll just start um can you see something yeah we can see it now we can see it now okay okay so huh i cannot move okay great so i will talk about the [Music] um the personalized algorithmic so i will talk about human error interaction uh within my creative practice practice and what i called uh personalized algorithmic storytelling um which i i paraphrased from eureka from uh from william or who storytelling how i understand it it is that it is a software-based ai based storytelling and algorithmic narratives grounded in users personalized data which influence the content and content becomes the form um and the software becomes the narrative gameplay and the plot um in order to make software and artificial intelligence visible and experiential in my work i let the technology agency fail only then we can learn the borders of the given technology and limited control of it i do this through balancing between immersion and alienation effects which compose the emotional scale and experiential design of each piece i will talk first about chomsky versus chomsky by um by sandra rodriguez that i co-produced um on a german side um i just to to give you the context i used to be a producer and creator in in germany in berlin for night uh i lived for nine years so that's why i was i was the german uh co-producer of this project and it was developed together with the national phone board of canada so ubuntu builder and i still film it premiered at sundance new frontier i wanted to share trailer but maybe because of the time shortage i want but you can sit in a in it google it chomsky vs chomsky uh first encounter is the first digital portrait made of the famous thinker let alone the one that uses his own theories of language to digitally create a chomsky ai his generative grammar theory gave the base to natural language processing after all the main subject of the experience is the user and thus chomsky ai is a case of a is a case study and a guide um we offer here with this experience which is a virtual reality experience a playful and easy access into machine intelligence um and we will rely on immersive documentary storytelling the distinguished the distinctiveness is of an interactive project where one can question ai by interacting with uh with ai um regarding the technologies uses in in this project chomsky ai uh builds on four sets of technology currently uh of four sets of technology so it's a chatbot system with a branching q a services so it's a q a maker a deep fake voice for which we use liability and intent analysis ais system lewis ai and and a four uh and fourth is the complex conversational machine learning ai bird we are also currently um experimenting with gpt3 um the data library used to feed the conversation draws from 5 000 questions um and 6 000 answers taken from public domain chomsky material and like all current conversational ai the backend system is also highly scripted after all the goal of the experience is not to create a chomsky version of siri but to support a compelling narrative and a fluid conversation with chumsky ai to do so we created a robust system where chomsky ai listens to users questions and then uses an algorithm to predict the questions intent and content before deciding which of the three conversational ai nodes um [Music] to choose from so so these three notes are first of all first it's a simple chit chat bot what most digital assistants do with static responses to generalized questions a control dialog flow following a narrative script inspecting user responses for intense tracking previous answers and determining next appropriate response and the third one is to draw from answers created by our complex conversational bot that uses natural language generation which is bird the ai system finally responds by transforming text answers to speech and emulating a chomsky-like voice deep fake not only do the users co-create through helping train the ai with their interaction and with their language but through using their tone of voice questions and audio inputs they co-create a unique sound environment and a musical score by an algorithmic mechanism which brings me to the next to the next project that i um co-created together with jamie balliou um and um it was inspired by the book the music by matthew herbert symphony of noise it's a vr is a sonic vr experience that uses sound and melody as a way to algorithmically create and organize the world around us the noises of everyday world are made into algorithmic musical score through interaction in virtual reality spaces the user creates music with sounds object and their body through interacting with environments such as movement and gesturing breathing and walking it was our aim to create sonic landscapes that are universal but at the same time highly personal the back end was done in the unity game engine and maximus p it's not i just want to make clear that it's not a per se ai um experience it's an algorithmic experience um however i want to i want to go a little bit deeper into the expert shells experiential design that i usually use that i use in my work which is a balancing act between immersion and alienation effects so um so in symphony of noise you enter space which you see on the right side that's that those are screenshots from the first uh from from the first world and the user uh blows into microphone when the bass is created and a bubble appears and it grows bigger by blowing into the microphone so um the immersion happens during through personalization and agency of the users so sound uh the users activate sound through controllers and through breathing however the alienation effect um happens when the machine takes over so the algorithm distorts um the breath into a bass wobble in world two um they the voice activation via microphone uh microphone creates creates melody so the user sings and speaks into a cave ice sculpture in in the space so immersion again happens through the personalization effect and agency effect which is basically the voice a voice activation and singing and the alienation effect is again voice distortion through an algorithm there are two more spaces fi and finally um in the final space the the the user creates um together with the machine um an algorithmic personalized score um um i'll just briefly introduce there are several other projects that i worked on so one of them was pre-crime calculator that we made in 2017 which was launched on itunes it's again um an example of algorithmic like documentary algorithmic storytelling where we used we basically simulated um and made our own uh predictive predictive policing software um to um show the biases and um basically the workings and functionalities of prediction policing software softwares and how they use and we base this on the interaction personal personalized interaction through their uh of the user through their own data and facial recognition the last piece the or the piece that i'm currently working on within the immersive storytelling lab which is a part of york university is called home is the world and i'm asking basically i'm asking the question um what is the relationship between human and ai memory it's failure and what happens when human and ai recreate memories together um yeah that's it for me thank you um this is my email address at york university um yeah i love to uh be in touch with you if you want to thank you thank you that was that was fascinating and i can highly recommend checking out uh the the videos the trailer it's really fascinating and uh to transition into um the discussion part i want to invite our distinguished moderators daniel and matthias to join us for the discussion and i'm pretty sure everyone has a lot of questions we've seen so many projects today that really span a variety of approaches to ai creativity working with machines being inspired by machines so it's it's really fascinating what we've seen today and i'm sure that everyone has questions so please also ask each other questions i will be addressing questions from the audience later on but yeah i also want to invite neil to join because neil would like to ask a question so i want to give the stage to neil okay i wanted to do that matthias and daniel's um kick off first but let me just kind of throw mine out there you know i i just want to say that kind of from a theoretical point of view um i think the importance of theory and what is theory but i think theory is about asking questions and i really think we need to kind of interrogate very precisely the kind of questions about creativity that we've received um i'm i'm uh much as i i respect margaret bowden i i'm i i think that her approach is could probably much outdated i think we need to kind of start challenging many of those things i think also particularly the emphasis on on the product rather the process i find a little bit um disturbing uh as someone who likes to think of himself creative i see it more based on a creative urge than trying to analyze the kind of the production or the strategies used for the production um so you know i i um and i think uh i forget the that they were the first person who was speaking but it was outlining it a series of different strategies that he put up together experienced playfulness and allergy construct analogy construction i i don't know the basis of those things i don't know the basis of margaret bowden's thinking actually on this is i think really we're in a stage where we need to really absolutely challenge something but there was something that i really found very intriguing um i i'm always struck by the fact that melanie mitchell says that ai can't be creative because it doesn't have consciousness i mean i i don't know that we are conscious of everything we're creating and i really like the comment that sir that um put out there that by roger penrose perhaps consciousness is after all merely a spectator who experiences nothing but an action replay of the whole drama i mean i think we're at a stage now i want to thank everyone for the presentation they are really collectively rethinking what is creativity challenging all these assumptions in the past i'm not sure that human beings are even aware when that kind of light bulb moment happens when these things appear maybe afterwards we think oh that's cool that's cool but i think we're really interested in this stage where we've got to kind of challenge many of these received assumptions because i think we're on a point where based on what i'm seeing today we can start to really um rethink what what creativity is but i want to hand over to daniel and matthias because i don't appear these these are the guys who've been invited today so mathias and daniel um thank you for the presentations really i was really struck by these things um really struck by them that's fantastic thank you everyone yes it's uh it was a very amazing like set of presentations and right now my uh my network my head is like spinning like crazy like how to process the entire information how to uh come up with a coherent in a way uh uh thought process in a way um so i really congrats like everyone it's really i think uh many of the presentations there is very interesting uh questions and uh i think also uh what uh what neil was saying you know i was also thinking about the same kind of idea when it comes to consciousness in uh in creativity and for me it's like is it important for consciousness to be creative in a way because we are saying you know we are saying that uh networks ai they are they're not conscious so that means they are not creative or something which is uh very very very sketchy for myself yeah it's a very sketchy way of understanding because if i'm thinking about consciousness consciousness yes is great as a human condition but in the same time it could be a very negative thing yeah um because if if we are thinking about ai uh we are talking about biases in aia but most of the time when we are talking about biases in ai we are referring to this kind of racial in a way biases or these kind of aspects but a bias is also a human condition yeah human condition is also biased for example yeah the way that you are encoding certain things that's also sort of bias yeah so for example if i'm very conscious about the process i'm going to encode that kind of consciousness into that into that network or the way i'm training things and so on yeah so what that means is that you know for example let's imagine a creative process so if you start to draw something and let's go back to drawing and i'm i'm i'm um i'm saying this based on a discussion that we had with briggs wolf bricks which is a um a great architect like a star architect uh and design principles at uh and we had the discussion with him a lecture uh on tuesday and he was mentioning this kind of idea of subconscious in the design process yeah so he was explaining in a way um if you if you are very conscious in the design process in the in the creative process what would happen is that you're drawing something let's say you're sketching something and if you are very conscious about what you're sketching suddenly you start to process and say okay uh i cannot do this because uh this is going to be too expensive uh i cannot do it this way because your is going to be made out of concrete let's say or this kind of issue structural issues going to come up so suddenly uh suddenly i cannot uh draw things in in that way yeah so consciousness in that instance then makes you be obedient yeah makes you in in advance in a way makes you act in a certain way and you know a very uh pre-described let's say way yeah it doesn't allow you necessarily to to go outside of your consciousness or outside of what you understand as being correct now and this is a very similar thing that happens also in neural network yeah i think manos was pointing to uh uh yeah vermicel manos bermuso was pointing to uh this uh three type identifies but identified by the misa sabes interpolation extrapolation invention and if you are thinking about it then if we are conscious about the design process or creative process mostly we are interpreting mostly we are interpreting within our uh what we are constant about yeah but we cannot extrapolate outside of what we are aware of yeah so is that then consciousness is that something that is actually important for the creative process or is something that actually just holds you back and doesn't allow you in a way to go outside yeah so i think this is something that probably i would also like to to hear the thoughts of of the presenters where do you see in a way this kind of this kind of idea uh this is a kind of idea of consciousness in the in the creative process is it something that actually helps us or is something that makes us be extremely conservative without actually realizing it yeah conservative in the sense of you know you're really like just working within what you're aware of yeah you're not going outside and you're not challenging in a way your own in a way understanding of things and then also i would like to uh bring the topic in of we are discussing about types of or creativity and in some instances we are talking about or some presentations talked about um ai being used as a sort of uh almost like an inspiration as a sort of muse and in other cases we are talking about the ai actually being creative yeah so here if we talk about creativity as a as a muse or ai as a muse or this kind of digital tool that we are using as a muse then for me that would be the question about like uh are we encoding our own uh human condition into this kind of algorithms uh we are too conscious about the process and then we are just encoding that kind of like very uh very conservative you know my understanding of things and then the other side of are the machines actually creative for me the big question that i'm i'm questioning uh currently is is that machine really creative if we are not addressing the the aspect of overfitting uh right now i know that we are most of us we are creative we are not so uh maybe attack savvy uh but um but that uh that aspect of overfeeding uh it's it cannot go together with the claim of creativity um we cannot really say that something is created if it's just overfitting if it's just memorizing a data set for example yeah um because that's the classical innovation is a room problem where just because you know to translate something based on a dictionary or based on a rule set doesn't mean that you are intelligent enough to actually speak chinese yeah so the same also here with networks just because you're able to output some images that look creative that doesn't make you uh that doesn't make you know a creative network uh just because you're memorizing everything and you learn based on rules in a way that so i would like in a way to maybe also matias will intervene now here but uh would be great after that to to to to hear some thoughts from from the presenters how do you see this kind of aspect of is is really consciousness that important for creative process or is it something that actually it's a strong bias that limits us to be creative outside of our own in a way understanding of things and then the other aspects of creativity when it comes to machines yeah thank you daniel thanks uh great comment actually uh and now after neil and daniel has taken away like the most interesting points i'm gonna try to pick up the scraps and see if i can do something out of it um first of all thanks a lot for the fantastic presentations they were amazing inspiring thought provoking i think there's a lot to discuss here i'm sure we don't have enough time to do like all the things that would merit actually discussion yeah but maybe some uh i would like to start with some thoughts about um creativity and you know how creativity and also how to how to basically approach trying to create a theory about creativity and the ai or maybe even just the ai and architecture in general because i think there is uh i do agree entirely with molly claypool here that there is no current theory on ai and architecture or ai and creativity in general there are bits and pieces and floating ideas out there and one of the main questions that come up is i mean i i have the feeling after rereading margaret bowden i i tend to agree with neil that some of the thoughts here might be a bit outdated considering the frame of conversation we have today certain parts however still true but you still have to think about it for example when you when you interrogate the philosophical framework of creativity you get immediately pointed towards imagination and originality yeah so there is a very faint discussion about creativity in philosophy but primarily those immediately go towards what is imagination and what is originality and that's a question really about can any i be a a can an ai have an imagination and b is it original yeah and uh one thing that is very clear here is the question also if we can actually rationalize creativity at all right is it or is it more like an emotion is it really a conscious process as needle pointed out it might also be an unconscious process is it a visceral reaction to a visual stimuli that some persons have and others don't yeah so there's like a ton of things here that need to be considered and um i also think that it goes along with questions that almost become esoteric like it's it's for me the question if an ai can be creative is like a question can ai love can any i hate can i sense fear yeah these are all sort of more more emotional status which i think are very hard to encode as a mathematical element that you can synthesize yeah so these are very very human questions at the end of the day now uh in order to understand maybe a little bit better like how do we basically discuss aspects of imagination i would like to point out to for example the writings of william blake who basically discussed that there's a dissociation of reason and imagination so that already points us to it towards the problem of of capturing these in form of a mathematical algorithm that allows us to synthesize it within an ai framework well friedrich schieler yeah who has had a similar thinking about especially the creative process and discusses that for example in his letters on the aesthetics education of men yeah and there's a variety of differentiations here about the the idea of imagination and creativity in terms of do you pick up on the knowledge of others yeah in order to build up your own imagination and creativity or as for example francis bacon pointed out everything has to be done from scratch yeah with experiments so there's like these two ways of of going about what actually creativity can be i think it it is for me as creative to pick up on historic examples of the architecture discipline and mangled them into a new project or trying to imagine them entirely from your head which by at the end of the day is not even clear how original that would be because i my argument is that architects are really good in synthesizing knowledge that they already have acquired over time and then produce out of that some sort of novel architectural solution there's also questions of representation for example a lot of the ai work we saw so today relies on representation imagery 2d representation of space right you can go along with with for example wittgenstein who actually what he thinking was that um alia kentness is a building so all knowledge is representation so the building of knowledge based on representational devices and if you think about the i or the neural networks we're working with they primarily work through that sort of problem like really creating knowledge through representation yeah um there's many many more points here especially like about how to create a theory like ideas what do i do do i use a deductive method do i do use an inductive inductive method etc so there's like a variety of questions here in terms of creating really a theory out of this but there's one last point i wanted to to go to a uh to my i was really happy that mikhaila pointed out the problem of bias because it was not mentioned in any other of the presentations and i was pretty surprised about that yeah that basically the idea and and this is really a criticism um that neural networks also have a tendency to have biases like in the way how we as the coders encode them with specific uh databases yeah and if you look into the work of ai in culture or neural networks in culture in general and culture for me includes painting poetry literature architecture etc like everything you can imagine because there is like in every one of those fields there is already neural work happening at the moment neural music for example the vast majority of those networks and results are extremely biased towards western culture yeah so they really don't mean the dominant feature within ai were currently is the western hemisphere and i think that needs to the the one potential that ai and neural networks has is that they can be extremely inclusive if the people behind it creating those databases allow it yeah and i think we have to raise the awareness that that that's an issue it's nothing that we should gloss over yeah um and i mean i'm happy about other these courses that are picking up on these notions which might come into the ai and culture uh real things like for example ruhab benjamins with her book race after technology and things like that or ideas from the feminist techno science that allow us to speculate about you know the the meaning of artificial intelligence within our creating within our creative industries and we shouldn't forget that there is something like creative industries and that they will be part of this discussion we are having today in a larger scale in the time to come and with that i would like to end my my like entry here because i think we they i would like to have discussion also with the presenters and yeah thank you very much for the opportunity to to make a comment about this amazing body of work we saw today thank you matthias um i actually wanted to invite to um to talk uh about the comment that he made earlier gabriele if you wanted to respond to neo because it was a very interesting discussion already starting uh in the chat and i kind of want to bring this into this discussion about pleasure um so maybe you can repeat for everyone the comment that you made um in reply to neil's comment earlier yeah sure neil was mentioning consciousness as something that differentiates us humans from uh machines and that's reason uh is said maybe for machines to not be able to engage in creative activities and i was thinking that maybe the difference is biological and the fact that the reason why we engage in creative activities is because we found pleasure in doing so and pleasure is something triggered by some very low level brain mechanisms and i think there is some kind of genetic reasons for us to find pleasure in doing something and that's something that we cannot encode in a machine not yet at least unless we can replicate the brain somehow in a synthetic way um yeah that that was my uh argument and i mean this thing was recognized uh by nicolas negroponte in the architectural machine because the very first idea of the blueprint for the architecture machine was to endow endowing a computer with all kinds of sensors right so what necroponte envisioned was a machine that could perceive the world through all different senses and what we are doing now we are channeling the the information like just by using just visual information and that's a very narrow view of the world so machines cannot experience the world they don't have any uh any way of seeing things the way we see them and this is another another thing for machine to not be able to reason the way we do uh yeah that's that's what i i said thank you i think that's really interesting because uh carfursten is also talking about that in his free energy principle that there is a still a very big difference between uh biological and um you know artificial systems and uh it's it's really interesting that he is basically pointing out that movement and attention um is is is at the core of any organism that it basically like moves through the world and finds attention and pleasure in certain things so uh we are we're certainly not able yet to uh encode that in machines but it's interesting to think that we might be able to um and i i kind of wanted to um bring that back and basically ask [Music] another question um so maybe yes sorry neil go ahead i just wanted to i thank you gabriel for that comment i think that's a really interesting comment um pleasure that makes me think of laura maldi's um essay on visual pleasure i'm sure there's something down there but maybe the death the term the term that pleasure is difficult to maybe it could be another um term like uh stimulation or interest you know and i i think these ideas have never been interrogated very much so i once had a discussion with andrew benjamin who was who used to be the big philosopher at the aaa in a few years ago and he said that doesn't interest me i said what do you mean by interest what do you mean and he couldn't tell me you know and i think this is really a a very very provocative question what is that whether it's pleasure or interest or stimulation i think you're on to the right the right to the right issues so thank you but here it's also i think um just to interject a bit here like if you say uh pleasure you're mostly talking about you as a human yeah like if i'm looking at at the output at the network let's say generates does great pledge or something but that's a sort of evaluation then as a human but you cannot evaluate the network from that kind of perspective is the network creative it's not created because i personally don't find pleasure there it's like you know that's i don't see it as a sort of measure of creativity dania because the same happens also with our work yeah like we have a very small pool of peers that actually find our work fascinating but the vast majority of the population of the globe maybe they see our work as being you know it was that just noise yeah so in that sense i think we we have to when we judge in ways or separate like are we talking about human creativity like uh or ai that helps us um to be more creative or uh inspires creativity in us or ai being creative yeah i think those are complete two separate separate things that we have to address them accordingly in a way after that i think my other problems is to say who judges who judges his creativity i mean did that with the fact that van gogh was not even recognized in his lifetime is creativity in the eye of the beholder in the mind of the of the so-called creative agent i agree with you yeah we should be able to differentiate between the product and the process because because the product is of course some it's a cr the creativity of a product is an extrinsic property that someone assigns to a product right so we are talking about the process and the driver the the driving force for someone to be creative in the first place so we are talking about motivation so we humans certainly great problems because we we find the pleasure to solve them that's that's the reason we make new discoveries because we generate new problems otherwise there would not be any reason to progress in in the sciences or any other other human endeavor we do that because we find pleasure in solving problems in discovering new things there must be a biological reason for that but yeah i think it's important i agree i think it's important to to kind of distinguish between pleasure as a kind of reward mechanism as opposed to pleasure as a kind of almost evolutionary kind of factor and i think this this goes back to the baldwin effect and you know which essentially describes the the fact that our genetic uh let's say change over time is partly encoded you know when we're born and partly depends on on the on the interaction with environments i think humans tend to kind of going back to you know reinforcement learning and the whole you know the similar concept right we tend to repeat those kind of actions which would tend to to increase a particular kind of um physiological reactions which are happening which in this case you know referring to pleasure give us pleasure and avoid other ones which would give fear pain or stress and and so on so i think this is kind of uh fundamental but also it also situates the discussion about the process of creativity as opposed to the result of creativity and and their creativity has been studied along different layers you know but i think at least on within the context of uh of my my work i think it's it's very important to understand the process and before we get to the point of the result because we can learn a lot from that um and i don't want to comment in relation to daniels and neil's earlier point about consciousness um with regards to our perception of that as a kind of more gestalt framework if we if we understand the value of of the creative moment as a as a as a kind of continuous process which which depends on several uh let's say modes of operation which involves both conscious and unconscious thought if you go back to poincare in the understanding that a lot of the conscious kind of organization of data or whatever preparation is taking place is only perhaps uh a small percentage of the of the internal procedure that goes on in our brain you know the unconscious process is much larger and the aha moment that is not really an aha moment it's basically a manifestation of the of the real of them of them of the moment where the conscious and the unconscious converge so we're actually able to to manifest that as a possible solution penrose has an interesting uh set of interesting comments about for example animal consciousness which you know aligns with the possibility of of the of this uh in in humans and the moment when you you you find that a chimpanzee discovers how to actually um reach for for for uh you know um as a group of a bunch of bananas which are hanging from a particular in a particular context and the the the interesting thing is that the the realization has leaves no doubt about whether the the solution to the problem that was being uh processed is correct and the same thing happened with with a lot of um historical studies in poncare himself talks about uh particular answers to mathematical problems which arrive after a long period of disengagement with the task at hand and yet what's what he's really highlighting is that he's extremely sure about the correctness of the answer there's no there's no uh you know let's say questioning about whether this is a possible solution but it's it's already kind of embedded that or there's some kind of inherent recognition that this is the actual solution to a problem and i think this is fundamental so ultimately we should not forget that you know just kind of to conclude here that the unconscious and the consciousness can only be examined in relation to each other if we if we don't have a conscience we can't say we have an unconscious and we could describe them as of course as semantic terms for us to reorganize our thoughts but i think at the end of the day even on a more physiological level if we look at the cerebral and the cerebellar distinction i think some some of these ideas become more more evident in the way that some tasks become embedded automatically and recreate it as part of our brain which again penrose discusses further yeah that's very interesting um martin please uh yes i wanted to make it uh make a point based on what gabriel mentioned about why why we are creative as a one possible explanation is because it give us pleasure i think we cannot compare human creativity to to machine creativity on the basis of pleasure because part part of the biological basis of pleasure is encoded in our behavior to reproduce and make we don't encode reproduction and mating in machines so let's talk for instance about birds when they have to find a mate you know to reproduce they have quite elaborate and creative behaviors to to attract a a female so so all we don't encode the the pressure or or the evolutionary mechanism in machine to to find a partner to make and to reproduce so so the machines don't have that perhaps if we encode that into them more elaborate ways of creativity will emerge right because the machine have to think in creative ways how it's going to attract a partner how he's going to reproduce how he's going to ensure that that that the the descendants they are going to survive so a lot of what we do in society has to do with with the behavior of reproduction and caring for for our for our children so so the when you talk about pleasure you have to take into account that pleasure evolved as a mechanism to to ensure that that we want to reproduce and and pass on our machines to the next generations and machines don't have that could i just chip in something this is a great discussion i mean i i personally believe that you know we need to just uh kind of isolate and say we're talking about human intelligence human creativity within a bigger spectrum you know i think let's not assume that we have the the oh we're in charge everything and i might personally believe in the notion of a second copernican revolution of of not putting us the center of things at the same time what i would say is that it'll be invisible to us you know in a way alan turing uh makes a comment about uh the machines can create uh poetry or and things and he said yeah sure they can but maybe only other machines will be able to able to to uh understand it so in some sense is a kind of creative and this goes back to the kind of famous move 37 that we didn't understand in that game of go up and go versus at least adults it was kind of beyond our comprehension so in some senses in some senses only it's kind of like we're limited by human our concept of human cryptic because otherwise we won't grasp that creativity coming out of the machine hi is it okay if i jump in yes please i was gonna say like uh ely and and you you had comments i saw your faces so please go yeah yeah okay yeah we're both from a current team so i guess for me maybe the more urgent question is that like when we define ai as a negative as an entity creative or not what utility does it actually have like what are the values it will actually bring to our social economic or planetary challenge by us sitting here talking about whether ai is creative or not i mean i'm so happy that victoria brought up um car physician who is basically just working at next door in ucl his free energy principle is so important not only because it sort of advanced on ai advancement but because his formulation of active inference the notion of any organized self-organized system would be able to use an internal generative model to minimize variation of free energy on incoming sensory data basically means that any living entity is has the ultimate goal to minimize internal or local entropy within the system so this basically ties information to energy meaning if we're able to use an internal generative model of some mechanical or social system we were able we would be able to understand energy transformation through information so in that sense if we're saying that creativity is also an act of minimizing entropy we can actually tie social economic value in being creative for the machine which perhaps accelerate the process i think that's also something that a current is asking very desperately sorry just a second now me first daniel no just kidding just kidding so i i have a um thanks a lot uh for that comment before i think it's a really important and interesting command it actually ties into what i wanted to to discuss which is actually i raised this point already at another i think the fiu phd conversations we have on sundays at some point i also raised the point whether um whether there is a point actually to discuss whether an ai is creative or not yeah it seems like there is like especially like in cultural theory there is not that much conversation about creativity as such i mentioned before that there is like discussions about uh originality imagination i've brought upset myself points like authorship agency and things like that which somehow are rotating around the the topic of of creativity but creativity is an enormously elusive term to discuss it's it's very very vague yeah it makes it very hard to to pinpoint actually the discussion about about it even in humans let alone in in a machinic process that we're using yeah so i'm wondering whether this is whether we're discussing actually something different here which is rather that the users of neural networks are encoding their own creativity within a computational framework it's not not the framework itself is creative yeah but the way how you label a database which kind of material you collect to get it done which images you're selecting to put in the database like all of that influences the final result that the neural network will produce and i think the creative process might happen afterwards when we interpret those results this was mentioned today several times in the conversations um martin garvino very specifically talked about this for example i think somebody else mentioned the architect in the loop yeah daniel also alluded to this a couple of times so i think this is rather the conversation that i think is happening here not too much with a ex whether explicitly the machine can be creative yeah but rather that what the result actually means within a creative framework that pertains to how we as humans interpret the resulting things and and i i have to admit that i heard for the first time the term men men in the loop uh when it came to the use in the military industry so they they basically for example automate drums drones that could do attacks yeah there was a man in the loop because it was important to understand what's the ethical implications of what the ai is going to do in this case and i'm pretty sure this is still the case today i don't think there is like little strikes that are completely automated there's still a man in the loop for the sort of ethical implications and i think that's the role we are playing here in terms of creativity we're like observing the results and understanding how can we use them within our own creative practice that's a great comment mathias i think right now i will be able to attack what i wanted to say before on two levels which is great just based on also your concept so this kind of idea of utility of ai like it was the utility of ai so here for example you're saying that we are encoding perhaps our creativity you know in this kind of networks yeah and that's why personally i was mentioning this kind of aspect of interpolation extrapolation innovation that the mississauga is talking about uh because if i'm saying i'm encoding my creativity into an ai i'm talking about that ai is going to create interpolations based on my constant you know my understanding of a subject yeah because that's how i am encoding things into that ai and then also addressing the other aspect of utility on of ai overall and creativity is like okay if i'm encoding a certain data that's going to be what what the ai is going to output so for me it's like why do i use an ai done just to iterate of course i understand there are levels of of creativity when it comes to interpolation extrapolation and innovation uh but mostly what we are talking about then we are talking about interpolations we are saying we are going to use ai to help us solve certain uh problems that that we have in the world like climate change or stuff like that but that ai is going to look only at the solutions that we input in a way or our understanding of the problem and understanding of what the solution might be so it's just going to create interpolation so um for me that that's why it's important then to think about is that ai actually creative is it actually going to be able to go outside of what i'm encoding in yeah and i'm interested then in ai mostly at the extrapolation level that interpolation level yeah is it going to be able to generate something new because then i see the utility of it yeah otherwise what i'm doing here i'm just really encoding uh uh like past structures and expect me in a way that i'm just going to interpolate i'm just going to every sun something out and i might find an answer so for me it's like also when it comes to climate change actually are we so sure that we as humans we know the answer to solve that problem so that that we are so confident that if we just input ourselves enough data into an ai the ai is going to be able to give us the proper solution for solving the uh solving the climate crisis or is it mostly more like can an ai actually shed light about a different kind of solution that we never thought of yeah you know i actually have a comment uh following up to that uh if it's okay but i think the idea of extrapolation versus interpolation is is really important um i think currently right now and especially with all of the the computer vision algorithms specifically with gans and regression networks that were discussed today uh they're all static you know there's they're trained there is this idea of training time where they learn features from this closed system this data set this conceptual space and um then they are deployed in the world uh but it's uh right now you know none of the algorithms that we've discussed actually uh deal with time which i think time is really important in actually successfully tackling this problem of extrapolation um and i think that's why it's really important to keep discussions of human create or like of human intelligence or human processing and machine processing separate because human processing is all built around time you know we all have spiking neural uh neural networks that are composed of spiking neurons um that literally uh encode information temporally whereas neurons uh within machine learning artificial neural networks do not encode time in the same way um and so i think that when we you know we almost need to kind of in this discussion of creativity say we can define creativity and simplistic terms for simplistic models like what we're working with right now and then kind of set a framework for maybe controlling how algorithms are designed and use time and extrapolation in the future for what we want to actually see come out of these models um i uh i actually there hasn't been a significant amount of work with spiking neural networks which uh are computational models that are more similar to how the human brain works but i'm hoping that as computers and gpu chips become faster and better that we'll see more explorations into how those models can be used for different computer vision tasks oh sorry victoria i don't we don't hear you but i guess manos or michaela sorry sorry uh just a quick interruption i just wanted to clarify because we're a lot of people talking so if you could raise your hands before you speak so that everyone has the opportunity to speak equally so please use the raise hand button before you speak sorry manos please it's just a brief comment because i was not in earlier when daniel was talking in relation to uh you know matthias's comment about what whether it is the the actual process of of you know ai that we are where we were using as a kind of vessel to to channel our creative thinking you know or if the ai itself can be creative and i think daniel i i agree you know with daniel 100 that the problem right now is that we can encode only our conscious understanding of of of data into the into a neural network we haven't found yet a way to encode the unconscious part and by that maybe this sounds kind of funny the way it's it's it's worried but what i mean by that is that um the ability to extrapolate in humans in relates and involves the kind of association that happens um you know that that is really not intentional on many on many levels so if we go back to um i think in my last diagram i showed the france rationale diagram which which i think everybody is familiar with which is talking about how unlike the classical uh programming structure of of ai of symbolic ai which inputs rules and data to get answers machine learning actually requires data and answers to give us rules so if we take this a little bit further i think the real way to to reach this kind of extrapolation which could somehow approximate what unconscious uh thinking or innovation from from a human standpoint could be would depend on a network suggesting the domains that it has to work within and not just constraining itself to work within the domains that we set it and by domain i'm referring to the conceptual space of let's say really interpolating between one kind of architectural group and another kind of architectural group but let's say interpolate between architecture and a completely different uh conceptual understanding like in the case of daniel's work on the sagrada familia you know the forest and architecture this was something that was encoded by by the designer but if the network could have suggested the domain then i think this would be a step towards you know making that kind of extrapolation start thank you um michaela please you had a comment um thanks for i would like to address alexandra's um the spike in neural networks question there is an app being developed in montreal um it's it's focused on sound and and and music uh but they're doing like a lot of work with spiking neural networks and you know art and and and sound so if you are interested in that i can glad you put in touch with them or just just just to comment on the spiking neural networks um second thing is that i would like to maybe divert the ques the the conversation a little bit less theoretically and um more pragmatically because i saw also the like in the panel on its own we went from like theory to practice which i dramatically really enjoyed and so my question um and also like regarding my practice like i i was one of the only uh and possibly there was some others maybe i missed it but i was the only who was talking the only one um and probably the current uh current as well who was talking about or thinking about their audiences and their um and and thinking about the users and what i know that some of the architectural apps were also of course you know it's um it's b2b so they were uh of course functional and and are made for for for usage but when we look at the you know audience and co-creation with the ai how users can do it themselves without necessarily being able to code like this is where i would like to kind of turn the conversation because it's not only about like the researchers speaking to their own kind and and just like you know the conversation is then too close right we need to get this conversation out and this is why i make stuff that might seem maybe for some people here pretty basic or um let's say commercial but at the same time it's all about how how can we increase digital literacy how can we understand how can people understand a.i and not be afraid of it and this is why for instance in my my work you saw the like the range from chomsky to like musical um environments or sound experiences and through pre-crime um calculator which is you know which discloses the bias of predictive software um in a simple way and so it's kind of you know going from co-creation to also you know bias and the datification of of uh basically our digital selves and i'm just surprised that like not many people have touched upon it so like that would be um yeah that's my question for the thank you um did anyone want to reply to that yeah that's something that we discuss a lot in current um from the for the last few years is this embodied real-time experience of the users and i think it ties into the conversation earlier about sensory inputs also the point that alexandra brought up about the relationship to time in the sense that we have sensory based inputs into the body like the way that all biological organisms feel versus the machine sensory inputs that are used for training the ai but if we can track the outputs back out of the body and um and and see the biofeedback relationship in real time then maybe some more interesting things can happen um especially if we're able to work with biofeedback interactions within artificial systems i think that that's a really important potential that we have there kind of like what michaela is doing with allowing the breadth of the user to interact with the ai outputs perhaps when we start actually being able to integrate real-time data whether it's biofeedback or big data from environments to train ai in real time maybe that's when we're going to start seeing more engaging and creative outputs the question is how and when maybe thank you that's that's that's very interesting um i i also wanted to bring in because we haven't had time for any questions from the audience yet but uh this is a related question from the audience uh what do any of the panelists question uh pandas think the role of the iii is in the context of society uh i see mine as a specific engine to give a perspective multiple engines can be called and mixed this is a question from austin cabrera on youtube so if anyone wanted to respond to that what is the role of ai in the context of society i mean it was kind of like you know with the audience you guys started talking about it so maybe miquela did you want to reply to that question someone else is uh their their hand raised from from previously did anyone want to respond to that yeah for right sorry yeah hi yeah i think um sorry my mind just blacked for once yeah i think ai wouldn't work as a standalone system it has to be an open system that breach with other types of technologies including decentralization that we see from blockchain including the architectural machine that someone also mentioned earlier with other types of algorithmic from procedural to root based system so it's important the question of how we actually bridge between these different technology systems so that they actually act as an architecture machine that is able to reproduce itself and prolong its sustainability and be able to evolve through time so like through different eras we would be needing different types of ai but if they're not open systems if proprietary efforts and open source efforts cannot integrate into combinatorial pipeline then the algorithm would just die like for instance um alexandria maquilla was imagining spike neural network if convolution and deep learning they're not evolving and breaching into other technology systems then spike is gonna come up and it's gonna replace them and then we're gonna have other super computers and then we're gonna have like quantum computing with exponential uh faster way to like decode stuff and encrypt stuff so i think it's a means of how to actually aggregate effort between different open source and you know even like um enterprise grade engineering thank you and since we are really you know reaching kind of a limited time here um it's very late in australia now i just wanted to give the opportunity to give last comments it's been an amazing discussion thank you everyone to all the great contributions that you've made are there any last comments or questions that weren't answered uh now's the time uh neil has it stand up yes uh yeah i i just wanted to i think it's been a great discussion i i think we should transcribe this and do something with it because it's really interesting i think we get we actually we're seeing things happening in the kind of in the chat not just like a brain at work and that's what i want to just throw in there i think some of the comments especially i enjoyed the comments by gabriel gabriel about the kind of pleasure all these things but so feedback is absolutely kind of crucial to all this but there's also a feedback out of the context and i think one of the real advantages of having a platform like this where you can bring people together from melbourne and god knows where you know is that you actually get this possibility of a real-time feedback going on and when it comes down to it i kind of get to get the sense that maybe whatever creativity is and it is a really difficult problem right that it might have something to do with that kind of interaction i mean that goes on in any in any sort of interaction itself so it's kind of like an emergent phenomenon i don't think we discussed this in my loss but i i just think that we are seeing now today uh creativity is actually happening uh i just uh before we end today i wanted to get final thoughts from daniel and matthias our distinguished moderators but yes um can go ahead i was waiting for you to talk okay um yes so all of you are leaving me behind with a lot of buzzing in my head like in terms of you know understanding digesting processing all that i've seen today and the conversation we had which i agree with neil was wonderful and it would be great to her transcriptase at some point maybe um it is hard to summarize all that was him today i think there are so many different voices here and i think this is one of the wonderful things about the development we're currently seeing in this area is that it's it's definitely not a form of style it's really more like a paradigmatic shift in cultural production at all in general yeah um you have might have heard the term neural music and neural neural art yeah like people neural art are people like mario klingerman sophia crespo and many many others neural music with databots and holly herndon who else comes to mind um jacht yeah so it seems like there's a real cultural phenomenon happening it's not necessarily a technological phenomenon in itself right because the the that we are sitting here and discussing is it now is the system creative or are we as humans creative it already shows that we're getting provoked by this new technology and that provocation is a thing a healthy thing to have you know in in a conversation about a novel paradigm of cultural production at large and we i'm happy that we're seeing this also in in architecture and it's really changing ways of how we have traditionally been thinking about designing and the design paradigm in general and i see that also of course in cultural production and we saw that today in several examples and i'm happy that this conversation was not exclusively about architecture but it was a very inclusive conversation about a variety of different cultural areas so all in all this is basically as i'm saying you're leaving me behind with a with a wonderfully bossy drunken head now to continue thinking about this so thanks a lot for these provocations guys daniel yes thank you everyone for a very stimulating uh session uh like i also started my comments early on like you know my network it's kind of spinning right now i'm trying to figure things out and so many great ideas here i just wanted to just address briefly on a last comment that was made about the need to integrate ai with other technologies and i think that's a very crucial thing and probably uh that's why i personally have a sort of position that i have when it comes to ai and this kind of being critical in the aspect of ai is just interpolating and just outputting whatever data we put in and just to point to uh to comment that memo act and made like a few sessions ago in a dds program he was pointing to the fact that we have to understand that the neural networks that we work with today they are mostly based on surveillance technology or they start probably to be developed that way and that's why also like mathias was suggesting at one point you know you have the western in a way stabilization let's say that it's very well represented in this kind of models and less other civilizations because mostly it's always you deal with this kind of issue of if a civilization let's say has a way to document itself more or has a technology then suddenly uh that that civilization will be over represented in a way yeah and i think that's a sort of problem of how we are what networks we are using and how how those networks learn yeah right now they are just these kind of massive collectors of data but yeah as we also discussed in the previous discussions like uh creativity not critically sorry uh learning it's quite different in humans than in machines yeah so i think this kind of i just wanted to pick on that idea and this kind of idea of we have to look at this technology not only uh in isolation but also how it's going to integrate with other technologies and also um what other developments can be done here and personally i think if we can move away from networks that needs uh millions of data's points uh the better because i think uh it's easier than also to remove these kind of issues of buyers and sonya so once again i think it was an amazing conversation i wished in a way and this is maybe something that uh digital futures can uh think of like i wish to to have this kind of discussions go on for for longer because there are so many interesting people here and we just managed to address i think a few a few of the presenters so that would be something amazing if we can think of a way to to open up in a way the discussion to have the discussion wait more longer than we have it now but thank you everyone once again it was an amazing session yes thank you everyone um neil uh you have the stage you wanted to say something uh neil you muted yeah just to say uh you know in terms of description of the discussion alongside the digital feature things we've got a doctoral platform where we're um discussing um things i don't think you can see this popping up but uh tomorrow we've got a session same time um where we have wool pricks and tom main uh famous architectural practices for those who don't know talking about how they're implementing ai in the office uh and especially we're also involving um uh daniel bolajan who's going to be talking about this amazing uh deep uh himalaya project that is uh just that is emerging um and and this is something what we're trying to do is establish a kind of a global classroom as it were that can where everyone from all over the world can kind of feed into this um uh so the discussion does continue um so i just want to thank people today this is i mean when we established this thing we had no idea that it was going to lead to this but i really think we're getting great if they keep coming out of it and it was a fantastic session like we're going to look at how we're going to go publish the transcripts and so on and uh i want to thank especially the guy from x cool um who showed us that we could in real time be having transcripts and of course we should have these in chinese as well why not um so thank you and thank you victoria and thank you as gustavo for all the energy put into it thank you also the background team who are supporting us a lot below the surface um and thank you also to to our special guest today amazing absolutely amazing um fantastic thank you yes thank you everyone and thank you everyone thank you thank you everyone thank you thank you everyone thank you for the invitation thanks thank you thanks everyone stay connected yes stay connected this is just the beginning hopefully there will be more connections maybe we'll chat on clubhouse okay

Channel: DigitalFUTURES world

Views: 127,415

Keywords:

Id: Gj-fFXz1kqc

Length: 174min 40sec (10480 seconds)

Published: Sat Feb 27 2021