Gopherpalooza 2018 - Ian Lance Taylor: Transition to Go 2

[Music] [Applause] [Music] and now yes we have the last talk of the day last but definitely not least in Lance Taylor from the go team and he's going to be talking I hope about generics but for today it's not that so I really hope there was generics so he's going to be speaking about something else which I should know but I already forgot cuz I got too confused with almost dying things to a drone the transition to go to so I'm still expecting to see some generics okay yeah I'm happy to be here at gopher Palooza it's been a great set of talks and thanks for inviting me and thanks to the organizers there's been a lot of fun so my name is Ian Taylor on the internet I go by the name Ian Lance Taylor I work at Google I joined the go team at Google in mid 2008 which was before the first public release of go so I've been working on go for some 10 years now it's been a great time and I'm gonna talk a little bit about it and we're gonna talk about where we're going in particular transition to go to so go was first publicly released in 2009 in those days the language in the libraries changed quite regularly for each new goal released we stay a week really weekly releases from any of them existing go users we had a few even from the start had to rewrite their code pretty much for each new release and we had a go fix program which helped you write your code automatically they knew about the changes we were making here's an example of what go used to look like for example this is about what it looked like about when I joined the go team in mid 2008 its simple hello world program so it's not entirely unger like but it's also not the same as it is today so it's changed that's my only point there in 2012 go 1 go 1 we said was going to be a stable release of the language and the standard libraries and that was going to give a stable platform for NGO users they no longer had to change their programs with each weekly release instead they could just develop and it would continue to work we promised that as best as we could if the program ran on one version of go it would run on all later versions of go and that's still true today of course we're still in the go one world and we still follow the goal one compatibility guarantee it's worth noting that this compatibility was made at the source code level we did not say that you could compile a package and then continue to use that package with future releases we said that you could recompile your package with each new release and it would work as it did before and there have been a few exceptions over the six years since go one but by-and-large has worked pretty well most people's go programs from six years ago still do the same thing that they did today which i think has been a good thing for the community and i think it's really helped with the growth of go to provide that stable platform last year however we decided the go team decided the community to decided it's time to move on Russ Cox announced it gopher con last year we were going to start a go to process for go-to we'll start considering significant changes to the language and we'll start considering the possibility of breaking changes and the same for the libraries will consider significant changes to the standard libraries and as you all know our most of you know there are potential big changes already being discussed modules is available as an experiment and go 1.11 we've already seen a good discussion of that today we've also talked about generics we've talked about changes to error handling and there's been a number of small changes and clean ups that have been proposed you can look on the issue tracker on github look for the go-to label you'll see people's ideas and the ones that are still open are the ones that are still under consideration ergo - which isn't to say we're going to do all of them we're not going to do all of them but we are going to make changes we are prepared to change the language that said this talk is not about those changes I'm not going to talk about generics here and I'm not going to talk about any of the other changes what I want to talk about now is sort of at a lower level we talked it we have to think about as we talk about these changes how are we going to do it how is gold going to move forward it's fine to say we're going to change the language but are we just going to do everything at once what's going to happen to existing programs how are we going to manage this transition as we move forward from go one to go - so when I think about this question what I think about is how do other languages do it goes not the first programming language it's not even the fiftieth programming language there's a lot of history we can look at a lot of past languages and let's learn from them just as we learned for them in the design of the language itself let's learn from them in the design of the transition of the language so looking at other languages as far as how they've made version transitions naturally the first language to look at is CEC is obviously a big inspiration for go it's an extremely popular language I'm not sure it's the most popular language today anymore but it's still one of the core languages of the world it's still widely used obviously the linux kernel itself is written in c many other programs when c was started around 1970 it was not the same C that we know today it had several incompatible language changes in its early years by about 1978 when the original Kernighan and Ritchie book was published just K NRC at that point pretty much all compilers had settled down even then there were several different compilers from different organizations they pretty much all settled down into compiling the same language and so C carried forward then they started a standardization process and see was first standardized in 1989 that's when they completed the standard meeting with what became called an CC and then in 1990 became called ISO C the standardization process added a number of new features to the language some borrowed from C++ some like the wide character support perhaps not so good or the multibyte support also some of the preprocessor changes I don't know but anyhow many of the features were good the standardization process did introduce some changes to existing programs I'm not going to go through this code here those of you who look at who knows C well can ponder the standard changed the integral promotion rules from being unsightliness preserving which they were before the standard to being value preserving so what the standard put in this was actually a breaking change it finally it silently broke some code code that looked like this silently broke when you switched from the pre standard version of C to the standard version of C this was not such a big deal in practice because most people did not understand the integral promotion rules and most people still don't understand that today so not very many people wrote code like this because they didn't know what was gonna happen anyhow but it's been okay and since the standard C has been completely backward compatible any code you wrote to the original standard C from 1989 will still run today it'll still work the same today C has continued to evolve not a lot but some what they've done it for example they've added some new keywords and the way they add new keywords without breaking any existing code is that the C language reserves a certain set of identifiers x' and your program is not allowed to use them specifically you're not allowed to use any identifier that begins with an underscore followed by an uppercase letter and you're also not to allowed to use an identifier that begins with two underscores so by using that what they do is they introduced a new keyword using an underscore in a capital letter which your programs not allowed to use then they introduced a new header file which uses a C preprocessor macro to define that slightly ugly identifier to a more convenient identifier for example there's underscore capitis underscore capital b ool is a keyword in modern c and then there is a header file called STD bull h and if you include that Center file it's just basically consists of sharp define bool underscore capital vol so you then get to use bool as a new type in your program but that can't break any existing code because the existing code wasn't using any underscore capital B identifiers because they are in the reserved namespace and it wasn't including STD bull H because I had her file didn't exist so that's an example of how C has been able to continue to evolve without breaking any existing code C++ another very popular language probably more popular than C today widely used all across the world C++ is actually the story is pretty similar to C it was first standardized in 1998 interestingly again C++ mates a few breaking changes in when they introduced a standardization process I'm not going to go through this anyone who's really interested in C++ can figure out what's going on now C++ has continued to evolve and while C has evolved slightly C++ has actually changed significantly over the years as they've added new concepts like move constructors and the library has grown enormously the standard library that comes with the language has grown enormously as they've added new features and as C++ has gone from the original C++ 98 to C++ 11 C++ 17 and they're continuing to work on it C++ has added new keywords to the language they usually add new keywords by picking um likely identifier it's like there's a new keyword called static underscore assert now if you happen to use that as a variable name in your program if you compile with the new version of C++ your program will fail to compile you'll get a parse error as it hits that identifier one of the ways that they avoid problems with that and this is true of C compilers also both C and C++ compilers have a - STD option or - standard option you can use the - STD option to specify the version of the language standard that you want to compile your program in so you can say this program is a C++ 98 program or this program is a C++ 11 program and by using preprocessor macros they also expose that request into the standard header files so the header files can also change behavior based on the language standard you've requested so by and large C++ like C that's done a very good job of maintaining backward compatibility for programs old C++ 98 programs will continue to build and run with modern C++ compilers and it is true even though --if is a C++ expert would tell you and I'm not a C++ expert a program written in C++ 98 the older version of C++ really looks quite different from a program written in C++ 17 nonetheless the language is quite compatible next language I'm going to just mention this more briefly is Java Java again like C and C++ have a very strict backward compatibility with each new release Java has a much larger standard library than C or C++ Java does have a process of deprecating features of the standard library they use a language feature annotation they can annotate functions with the app deprecated deprecated annotation and you know then the function was theoretically deprecated and then in principle they can then remove the library from future releases of Java I'm not a hundred percent sure whether they've actually ever done that but they can and so Java is also very backward compatible at the source code level I'll just briefly mentioned that Java does have kind of a different issue which is a kind of a forward compatible compatibility issue because Java programs execute on a virtual machine the JVM the JVM bytecode z' sort of constrain what the language is able to do as it changes because the bike codes are at a much higher level than ordinary machine op codes you can't change the language or rather let me say let me say that again in order to change the language in a way that the byte codes do not support can require pretty complex changes to the JVM and historically the java language has avoided making those kinds of changes that's a interesting restriction if you worry about the java language it does not apply to go because of course go compiles to machine code except for the new web some bleep or and you know therefore go can do in principle anything the machine can do which is pretty much anything so what have we learned from looking at C C++ and Java these are the things that I've learned backward compatibility matters these languages are all successful and these languages are all backward compatible new keywords are okay at least sometimes and the reason they're ok is if you build with you know the new version of the language that supports the new keyword you program won't compile it's not great it's not perfect but it's not like it's gonna trick you there aren't gonna be any isn't your program isn't gonna mysteriously change to act differently as the C and C++ examples I showed the priest standardized ones had the possibility of acting differently silent behavior changes are risky and since standardization C and C++ and you know Java for its whole history have avoided them we don't want a program that works one way in one version of the language to work a different way in another version of language it's one thing if it fails to compile at least then you know you've got a problem even if you don't know what it is if it compiles and acts differently you may have a transition problem and both C and C++ show that it helps to have an option to say which language which version the language you're compiling for these are the lessons I've taken from those languages and I'm going to talk about another language Perl so the so Perl you know Perl was a very popular language at one time sorry I don't mean I don't mean to dis Perl I've actually written a lot of code in Perl it's a great language Perl was a very popular language at one time and it went up to version 5 and then in the year 2000 development began on Perl version 6 and the first stable version of Perl 6 was available in the Year 2015 which is to say 15 years later Perl 6 was intentionally not backward compatible with Perl 5 it was intended to be based on a spec rather than an implementation the earlier versions of Perl had all been based I mean there was exactly one implementation of Perl 1 through pro/5 it was written originally by Larry wall a lot of other people contributed to it and you know if you ran that source code whatever it did with your program that was Perl so for Perl 6 they said let's try to actually specify what's going on which in general is a good idea Perl 6 the process began with a set of change proposals all ideas of how to change Perl and then it sort of continued to evolve and then they really continued to evolve and 15 years later it was available and in the meantime many of the ideas had migrated back into Perl 5 and you know the length of time of that transition that was a problem for the language now I don't think pearl is nearly as popular today as it used to be and it's not all the fault of the transition other languages PHP Python took up a lot of the mind share of Perl but I do feel that at the time when Perl was at its peak of popularity they made a decision for good reasons to transition to a new and incompatible version and they lost mind share which might have happened anyhow a might not have hard to say but I don't think it helped let's put it that way Python Python an extremely successful language it's more successful than go clearly more popular a lot of people use it and it's a great language in the Year 2006 they started the Python 3.0 process also called Python 3000 Python 3 was first released in 2008 which is to say 10 years ago now the transition process from Python 2 to Python 3 is still going on it's winding down most people are making it to Python 3 but you know I work at Google Google is actually a big Python shop a lot more Python code at Google than go and just I think it was just two months ago that they said you know okay no more Python to code your existing Python 2 programs are going to stop running in the next year so the transition isn't done although it's been 10 years since Python 3 came out so Python free was not backward compatible with Python 2 it was pretty similar but some things changed the print statement changed notably strings were changed to use a Unicode representation and Python is often used in conjunction with C often you know your python code calls out to see for the bits that are really performance sensitive the change in the Python string representation meant that if your code passed any strings back and forth between Python and C which obviously a lot of code does you had to rewrite your C code so that made it harder change to Python 3 also you because Python is an interpreted language and it runs inside an interpreter in the interpreter you are either using the Python 2 interpreter or the Python 3 interpreter you couldn't have a program which was a mixture of Python 2 and Python 3 you had to pick one or the other and Python programs just like programs in any language you use a lot of libraries that are written by other people those people are on their own support schedules if you wanted to move your code to Python 3 you couldn't you could obviously you could write to the common subset but you couldn't actually switch to Python 3 until every library you depended on had switched to Python 3 and so libraries themselves couldn't switch to Python 3 because if they did they break all the people who use them who couldn't yet switch to Python 3 themselves so libraries were stuck maintaining both python 2 and python 3 versions for many years and some of them are still doing so so now I can't see my notes here so all these issues actually had nothing else was saying all these issues have really um slowed down the transition from Python 2 to Python 3 it's still a successful transition it's still a successful language but it's been slow and I don't think they anticipated before they started quite how slow it was going to be so what have we learned from Perl and Python compatibility earlier I said compatibility matters compatibility matters a lot if you're going to transition forward the lack of compatibility really slows the transition to new language versions and it's really good if you can link your old libraries and your new libraries together if that is to say if you don't have to wait for your old libraries to upgrade to the new version of the language before you can start using the new version of the language for your code that's a feature that's C and C++ and Java all have but Perl and Python did not have so that was a very brief tour of some popular languages and there's a lot of other popular languages out there which I am NOT going to talk about today let's go back to go now go to how can we apply these lessons to go backward compatibility is really important we can safely add new language and library features to go that's not going to break any existing code that's fine it's includes you know the big features most people are talking about for go-to we can add generics it's not going to break anybody we can add new keywords that is going to break some people people who use those keywords for their identifiers but it's not gonna cause any silent breakage when old code uses a new keyword and is mean that is used as a keyword uses of identifier that's a keyword in the new version of the language and you compile it with the new version of language that code will fail to compile the error will point right at the keyword it's not the best experience and I'm sure we can make it better but you know the developer is gonna look at it and let's say you know oh darn it I've got a rename that identifier to something to get around the problem and of course back in the go zero days we had to go fix tool which updated your programs for you we can provide that too we can have a go fix tool that I mean if we introduce any new identifiers which is not a given I mean any new keywords sorry which is not a given we can have an a go fix tool that goes through and renames all your identifiers that happened to be the same as keywords and carries forward for you so we can't do that we can also safely remove a few language and library features not that we should look to do so but I want to make clear that it is possible even though in c and c++ have not done so we can remove them provided that we're moving them breaks code that uses them we can't have any silent changes and provided that the workaround is simple very simple and provided that this go fix program could do the change for you if you choose so all those things if all those cases are true it is possible for us to remove features now most other languages don't remove features so this is a risky step we should only consider doing it for cases that many programs get wrong and we can only consider doing it as I've said if removing the feature would break any programs that use the future now why am I going on about this why don't we just keep all features here's an example with a feature we might want to remove because most people misunderstand it it turns out that go since you know the very early days has had this feature where we can convert a number to a string and people do use it once in a while I think oh I can convert a number to a string it's so much easier than calling stir coffee I to a or anything only it turns out that converting a number to a string actually treats that number as a Unicode code point and returns a string with the utf-8 representation of that code point which is occasionally useful but not something most people expect to happen which is why if this program if you run it will print a little smiley face character and it will not print the string nine seven eight six so you know I bring this up at some length is an example of a feature we might want to move now if we did remove this we would just remove the ability to convert into string from the language and most programs wouldn't notice the programs that do they would simply change to call the appropriate standard library function which is more or less utf-8 dot encode room although that one uses slice bite instead of string so we would have to adjust that slightly and we can do even better because we use the go tool to build go code the go tool can know which version of the language it's using to compile your code obviously so I believe we should add a feature to the go tool that whenever it builds a package successfully built a package it should record you know in the go-to mod file that we have now is part of the modules experiment assuming we keep the go-to mod file it should record in there which language version it used to build the package when someone else does it go get of the package they'll pull down that go mod file they'll see which version of the language was used to successfully compiled the package and then they will compile the package using that version of the language now to be clear what I mean is for example if you compile your code with go 1.20 and in go one 221 remove this feature and someone running go 1.25 you know years later does it go get of your code they'll build your code using the goal 1.25 compiler with a mode saying use the go 120 version of the language just as C and C++ compilers have a - STD option saying which version of the language you want to use to compile the code we can have an option saying which version of the language we want to use to compile the go code we can do even better because we can record that option when we successfully compile it and we can use it automatically as other people go get the code so we can continue to so we can wear appropriate remove language features without breaking existing programs because our tooling can automatically do the right thing which i think is the way to go of course if you do use this and you do want to use new language features then you're still going to have to fix your code just to be clear all I mean to say is you can have old libraries written in old versions of language you can link them together with new libraries written in new versions of language and everything will work as expected an important point that other languages have shown of course there's also libraries for libraries I think the modules experiment shows us the way forward where it seems useful we can introduce a new version of a package that breaks compatibility with the old version of the pack by introducing a v2 of the package or if we really get it wrong we can go on to v3 and so forth the most important point here is we'll have to ensure that version 1 and version of 2 of the package can both be used in the same program a classic example here would be the net HTTP package is very easy to imagine that we would want a net HTTP v2 but you might have other packages in your program other libraries that you're using that you rely on net HTTP version 1 we have to be able to link all those packages together some using in HTTP version 1 so I'm using HTTP version 2 they all have to work together in one big happy go program and that's not impossible because we only have source code compatibility so when you you know if go 1.20 has HTTP version 2 and you gonna have to recompile all your code with go 1:20 to work that means that you're going to be using the go 120 version of HTTP version 1 so when we release version 2 we can ensure that it is interoperable with the goal 120 version 1 perhaps by having them both call some internal package perhaps even by having the version 1 code call the version 2 code why not so it's not going to be easy it's going to require thought and careful design but we can move forward to version 2 of packages and continue to have old code work if we do that maybe we can have a deprecation plan for version 1 packages maybe we can say ok it's been five years time for that package to go away or maybe we'll keep it forever it's hard to know at this point what's going to be best also it's worth mentioned using modules may allow us to release patches for packages without having to wait for a new goal release and we could perhaps move some standard library packages out of the standard libraries so we don't have to ship them with each new release they'll still be available via Koga so that's our path forward to go to the most important thing about the transition process is we don't want to break existing programs that's what we learned from looking at other languages we want to be compatible we can add new features we can perhaps remove a few old features we didn't want to minimize that we can add new packages with a different API we're going to keep the old API around the lessons we see from other languages is that the languages that are most accessible over the long run do not have breaking changes there is no C version - there's no C++ version - there's just the same C and C++ they keep moving forward you know they keep changing but there's no dramatic change so with regard to the overall ecosystem go-to could be considered harmful what we're really talking about is not the transition from go on to go to is the transition from goal 1 to newer better versions of go one maybe will change to version number two at some point as kind of a marketing move everything will appear shiny again it'll be cool but your old code will keep working so that's the go tour transition it's an exciting time for go I think but you know what I've been doing going for 10 years it's always been exciting and it's still that way so it's been a great conference thanks everybody thank you for the great talk in I wanted to make sure that you all got that fun of go to might be considered harmful that was a great pun so yes I just wanted to make sure yeah thank you for that fun that was a very good fun so [Applause] [Music] [Applause] [Music]

Channel: Naut Human Productions

Views: 14,520

Rating: 4.8888888 out of 5

Keywords: golang

Id: LqKOY_pH8u0

Length: 33min 24sec (2004 seconds)

Published: Wed Oct 24 2018