Azure Managed Instance for Apache Cassandra and Cassandra API for Azure Cosmos DB - Episode 26

[Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] hi everybody welcome back azure cosmos tv live tv uh i'm your host mark brown uh episode 26 we're going to talk about something brand new to our show that we've never talked about before and we've got a brand new guest to our show theo how are you hello i'm good thanks how are you mark i'm good i'm uh suffering a little bit from the pollution around here i don't know what why i don't know pollution levels are elevated it's a bit unhealthy and i'm a little sensitive to it so i've been kind of coughing a little bit uh here and there trying to try to wade through it so um heard you had a little bit of a cold there the other day hopefully yeah yeah i'm better covered i think i'm surviving so yeah how we go that's all right that's all right so we're going to talk about uh a topic we've never talked about on the show which is uh cassandra api so for i think most most viewers know we're a multi-model database service so we support multiple different database apis and one of those is cassandra uh you want to talk about i mean like why what is cassandra uh why would why would why would i use it uh maybe that's a good place to start yeah um i guess shall i shall i do should i share my uh do you want to why do you got some slides to talk about this too yeah yeah and uh yeah hold on let's go some this is something that i threw together ah okay yeah so so um yeah i mean i guess i guess um you know folks watching uh many of you will will know what cassandra is but some of you might not so i mean i guess these are the cool bits uh for me um cassandra was the backbone of facebook that's where it started uh and netflix as well so that those two things alone i guess are a pretty uh cool thing to have in your locker um it's it's a has a fairly short learning curve for app devs uh um cql which is the language is kind of a subset of sql i guess and it's really it's strengths are um it's you know it's very um highly distributed very linearly scalable uh it's very right optimized with fast reads as well um it's very resilient and fault tolerant and it's really a multi-master uh system by default there's really no single point of failure in cassandra and in a lot of ways it's kind of similar to cosmos db and how it's architected at the back end except it's really kind of hyper focused for for performance and availability um and it's super configurable um at the platform level as well so there's lots that you can do um with consistency and replication both on the right path and the read path as well so you can really get in uh you know um go to town on on those aspects i guess i wanted to ask some questions if you could go back there so yeah so facebook if i remember correctly was a heavy note uh mysql shop i think right so did did cassandra kind of grow out of out of their frustration with with mysql or was this something kind of completely different yeah i mean as the story goes as i've heard it um they got to a point where they couldn't scale they got to the scaling limits of mysql so they basically built cassandra so the cassandra project started in facebook eventually they open sourced it and became apache cassandra okay so yeah they're nodes there so they they run cassandra's got a notion of a cluster and then you've got different nodes right and those do those nodes all basically are they copies of the same data uh within the cluster or the group no the they're uh your pieces of uh very very similar to how cosmos db works so you've got um uh copies of uh the data in different places and depending on your replication factor and so on so they've got partitions yeah physical partitions of data yeah and those roll up into replicas and then you can have multiple copies of the replicas in a replica set so their their topology is similar is that what you're saying it's similar yeah they the the default by default or i suppose uh by recommendation it would be a replication factor of three so we default at four but the the setup is very similar the approach to replication is very similar and consistency is also very similar um what what you have is a lot more configurability on the consistency side in cassandra really that's the main difference i see i see so yeah and we've got what we call partition keys they those are cluster keys is that what they call that or is that what they call it well they call it um primary keys which is kind of confusing if you come from the the relational world yeah um but the combination of a primary key and a clustering key can be a partition key and also a primary key on its own if you don't have a a clustering key and the clustering key is if you want multiple records per um partition if you like they don't call it that but um you know the primary key is basically basically the partition key yeah i see i see okay yeah sorry to stop you i just i there's some things i know about cassandra but i've never really i've never really gotten hands-on with it and i only kind of had a hand at arm's length understood kind of their architecture so yeah you probably know most things about it if if you know uh cosmos db it's there's a lot of similarities um but it really kind of tilts in that direction on on performance and scale and just really fault tolerance and um you know availability um but then i guess on the other side of that coin what's not maybe not so cool uh is that you know it is highly configurable but it's also complex to maintain um i would say cosmos db is pretty obvious really it's a lot easier um and and there's not really much of a query engine you can't do many aggregate operations um it's very very sort of you know geared towards atomic things uh so sql api even sql api which is also you know oltp focus but it's a lot richer in what what it can do in its query language um and the and the yeah the fault tolerance and the resilience maybe this is being a bit picky but uh i think it it can be a bit too flexible you you can set any replication factor you want including one which means you're not really you know uh it's hard at all so it kind of you know all bets are off if you're doing that kind of thing and also when you've got something as configurable as this and let the levels that you can go to both on the right path and the repath with quorum settings and hinted handoffs and so on um you know you almost need a phd in distributed systems to know how to do it properly you could say whereas something like cosmos db has these five well-defined i would call them kind of business use case oriented definitions for consistency right so it's quite easy to understand when you when you go through it um you know um consistent prefix and session consistency and so on there they're easy to understand you can understand the use cases that they fit uh but there isn't anything like that in cassandra it's just very kind of low level so you have to really intimately understand um how how the platform is working to know whether it's fitting what you're doing so that i mean you can look at that one of two ways you can say it's really configurable and you can you know thread the needle on things but you can also say yeah you kind of you you know i looked at the consistency so we have a dock that maps cassandra consistency right yeah and i'm i i we spent a lot of time on that and and in the end there are there aren't actually any exact analogs um but it's helpful to see you know what the closest analog is um but but you know even even there that's that that was simplifying it to be honest so uh if you really were getting into um cassandra then there's there's many levels that you can go and i my personal view is you you don't you know it's quite rare that you need to go go to all those levels but um you know who am i to criticize designed by someone that uh likes lots of knobs and levers i guess yeah well i guess facebook is the kind of sort of the perfect place actually i guess it does it it reflects it's uh the dog looks like it's owner i guess yeah right that's kind of a weird metaphor but yeah interesting yeah so i guess that's cassandra so yeah i mean this t's up so why why us i mean i think you kind of spoke to that yeah yeah i mean i guess most of the viewers will know uh what cosmos db is at least at a high level it's a managed service right has a lot of um things that you can't get if you just deploy a database like cassandra in a single you know a single tenanted way um you can't do things like auto scale it's not a multi-tenanted service you can't do things like serverless and you don't have these guarantees with slas and so on and so forth so you know we could talk a long time about all the benefits of cosmos db um and then what is cassandra api it's pretty obvious it's it's a compat layer over cosmos db so what we do is is transform uh requests that that meet the the wire protocol for apache cassandra into what we need at the back end of cosmos db and then we we respond with what is expected with the y protocol so the application that's designed to work with cassandra uh can work with cosmos db through the cassandra api and we're basically pretending uh to be cassandra uh at the wire protocol level which means you can just use any of the open source drivers that are out there and talk to cosmos db and get all of these back end uh platform benefits by just changing a connection string and pointing to a you know a different endpoint of course that does depend on uh you know uh whether we uh support the feature um and we to be honest we we're pretty close actually now to supporting uh most most of the features that you would expect um let me find my the right window here what you probably want to do if you if you're watching this and you um you're if you know you're looking thinking about a a service like cassandra api and you maybe you're evaluating it you definitely want to go and look at this dot first i don't know if we get the link up there this is our kind of main support dock and it it details uh everything that we support and don't support uh in cassandra api um that maps to the features that you would find in apache cassandra so do we support multiple versions of cassandra we don't no we're kind of sticking to the uh sort of evergreen uh paz concept if you like so we don't we we support all of the features uh um uh you know going back to uh an earlier an early version of the protocol it's just it's we don't take the same approach as we've done i guess in in mongodb and even in there um you know like three six for example could contain some 40 features and um 40 will probably contain some 4.2 features and so on so we we take it more from a um uh you know a paz service not being tied to uh releases and things like that which is why we detail all of the features here that we support so it's kind of works for whatever version you're looking at you can look in here and say what is what is my uh what am i uh what's supported here got it got it i got a question from a viewer uh is there a monitoring tool or a way to get replication delay between geo-replicated nodes a way to get replication delayed well you know what there is the probabilistic bounded staleness yeah in our classic does that work for cassandra if you're using cassandra api or yeah where at the database level uh it works what i will say is that you can't override um the consistency levels in the same way at the client side or at least if you look at the mapping document it's it's a a little bit more basic um but at the database level yeah all of the consistency settings work as you would expect and in the manner described and the monitoring of that too right it's like i think pbs the i hate saying this word because it just rolls off the tongue but probabilistic bounded staleness is the metric for replication latency yeah in cosmos and it's right now buried in our classic metrics in the portal and i know it works for sql api i'm guessing it works for our other apis because we're talking about at the at the data replication layer this is well below this is at the storage layer well below the api layer so i don't think it depends on which api you're using but that's no yeah it works it works in cassandra api just the same yep yep great yeah cool this is quite a oh there you go that's my uh here let me let me uh go full horrible [Laughter] yeah my favorite uh doctor like what is all these things i have no idea local one each quorum whatever local cereal what uh yeah so uh i mean you know we have more than just strong and eventual uh we have five uh but we're not this is the way this is way too much um you can a lot of configuration you're i think you're right i think you're on the phd piece of it there so yeah my advice is uh rather i mean we put this in because we had asks about you know how does this map to cassandra but my advice if you're looking at this is honestly just look at our docs on on the consistency levels they're a lot they're actually a lot easier to understand and just think about how you would adopt those settings and how those how they fit to your use cases that's honestly easy thanks for saying that i wrote those uh so yeah and i still uh uh i mean they those docs were pretty dense and hard um before i went and re revamped them i tried to make them more relatable uh people still ask me questions like i don't understand but i get it distributed databases can be a tricky thing and understanding things like consistency models and consistency levels as we call them can be that this it just is not intuitive in some ways yeah distributed systems and distributed databases are kind of unique it seems ironic doesn't it that we that we um actually make it a lot easier than another database out there well you know what i mean that's why we have so many customers uh particularly you know like a lot of azure services that have to run they they are by definition distributed systems they run on a cloud and in multiple regions and that's why so many services in azure take a dependency on us it's because we've solved a lot of their hard problems yeah and made it easier for them is it is it completely friction free well no there's still there's still some things but yeah all right so some new features uh you've got here um yeah so i wanted to i wanted to show that that the support dock showing uh you know the features that we support and just to mention um you know of course the these features actually i'm showing here will if you were to look at that dock and look for them uh they would probably be listed as not supported because most of these are in private preview so i just wanted to call them out if you are evaluating cassandra api and you and you look at that dock and it's certain feature is not missing then it might be here it's always worth asking or reaching out to your account team or your friendly microsoft person uh that you're in contact with uh to find out if we're building something we're always trying to add uh more compatibility and of course also unique features that we expose as well but these things that you see here materialized views uh lightweight transactions uh with condition support we already support lightweight transactions but with with conditions uh truncate um uh and also support for glow root which is an apm tool um uh these are all in private preview so if you use glow root or if you uh use these other features you should reach out um and uh um if you're evaluating this and that's a missing feature for you then then please do reach out and we can materialized views so earlier something you said caught my ear and that uh they don't really have support for doing things like aggregates like that is that is materialized views apache cassandra way of getting around kind of limitations of cql itself uh because yeah use those to basically do aggregates that create materialized views out of them is that is that it's it's exactly that although i would argue that our implementation is going to be more more efficient um maybe i won't go i won't go down that road right now but but uh because it'll take too long uh um to describe it but but yes that's that's basically the goal of it it's actually a feature that was experimental in cassandra for quite some time um so you can that kind of tells you uh something about the difficulties in in putting it into cassandra but uh you know of course for us uh it's more straightforward because we have a change feed and so that's the basis for how we've built materialized views and we we expose it exactly the same way uh as you expect to see in cassandra so it'll work from a you know from an api um programmability perspective um but yeah you should be um pleasantly surprised uh when looking at that so i put this uh url up for the cassandra change feed sample is that uh good place to put that up there i see you've got your yeah are you just for change feed up there you want to talk about that i guess yeah so there's a couple of i mean i will uh these uh you know native cassandra features they'll be well known to people who know cassandra but there's a couple of features we're adding um that are unique to cassandra api so you wouldn't find this exactly in apache cassandra um and one of them is change feed we already have change feed we've already exposed change feed in cassandra so just i guess for people who who don't know what change feed is or maybe you missed tim's great uh session that he did with you uh a few months ago um just very kind of high level you should definitely check that session out as well because that will tell you a lot more about how this works but very high level uh any database if it's you know there's data coming into the database and you're going to want some way of persisting it obviously uh and tracking um the changelog in some way so the way that we've done this is with a distributed uh event store which we call change feed there's various analogs for this in different databases cassandra calls it a cdc a change data capture um and other other databases will refer to it in a different way but what we have which is a little different is a um a programmability model where you can subscribe to changes that happen and then read them and then use use that uh to push data to other targets and you use it for event sourcing and and so on and so forth and we've exposed this uh in the cql language in cassandra so you can read it directly from cql so um it's a query predicate in a in a cql query in cassandra and basically you you um you run the query and then you keep setting the paging state effectively it's just a query that runs uh forever right because it's a continuous uh um feed of changes um uh so that's that's how that works and there's obvious there's obvious uses for that um one of the things you're saying in there so because this is it's confusing to me i guess at some level because for change feed in our our sql sdk it's basically you just set up a delegate that sits there and listens yeah and then that fires every second or whatever interval you set and then more data appears in a collection or an array how does this work and you put this on like a while oh i see it is on a while loop there yeah okay this thing just sits there kind of hanging just in its own thread and then it somehow that acts as a delegate this cql if you will or yeah well yeah this this is how we've had to expose it so we're exposing it as though it's a query but we're actually interacting with the change feed um the reason we had to we're doing it this way uh just to kind of expand on what you're touching on there is um of course in sql api uh we we control uh the client drivers so we can control how we interact with the change feed and the programmability model including things like change feed processor and how we do um you know um [Music] partition key ranges and continuation tokens and etc uh we don't have we don't have the same control obviously over over the client drivers in cassandra so we've exposed it in a different way uh and you you query the change feed as though you're querying a table but you're actually um interacting with the feed the never ending query so you're constantly updating the paging state to see what is available sometimes there's nothing available if there's a change there it will be available so that's how we've exposed that now that's very that's ingenious uh theo whoever came up with that give them a gold star that's brilliant because you're right you don't have to worry about compat with a driver you don't have to roll your own driver yeah for this thing it all just works and you're staying true to the spec if you will uh by just having this special table that you're uh running a query on so okay very cool that's uh man i'm getting smarter uh sitting here today this is great so yeah i'd love to say it was me that thought of that but it wasn't me but i agree it's a very good idea it's a great idea i love it yeah so what one of the uh limitations with that however um is if your client is um you know querying uh the gateway which is what you transforms your requests into what we need at the back end um at the moment we haven't exposed this uh as something that can be parallelized so when you query uh when you run this query uh the gateway sends requests to all politicians and we're getting everything back so if you have hundreds or thousands maybe uh of partitions then your your client is really acting as a bottleneck uh in that sense um so what we've done uh to kind of bring this more in line uh with where a sql api is is exposing something uh called which we calling feed ranges and these feed ranges you can think of them like a token that you that you can use to parallelize uh querying across um the whole change feed um and they they represent uh the whole value space that's possible in a table if that makes sense so um if i um if i query a feed range um it can still handle splits in the data at the back end um i don't need to um you know be aware of new uh subsets of feed ranges that appear so just maybe this will help to illustrate this if we uh if data grows uh then we'll add new feed ranges uh but they won't actually be new feed ranges it will just be the same data divided into a large number of chunks if that makes sense so what i'm saying there is the existing feed ranges will still work you'll still have the same parallelism as you had before uh it's just that you know there are more uh feed ranges available uh you just have to restart um uh the client to pick them up in in this case so that's uh yeah and just to clarify for folks that may be confused so with change feed change feed actually lives on the physical partition and so that's why when you see things like the lease container that you cannot do for cassandra they've got their gateway and you need a way to be able to parallelize the consumption of the change feed that's across these feed ranges because they live on different physical partitions so that's what you're looking at here if i'm gathering this correctly yeah interesting okay that's uh that's an interesting uh uh that's interesting then although you guys uh went in and tackled that as well so okay um oh you got a demo yeah i was going to do a little demo of this um just i got a question from a viewer maybe um since we're talking about kind of physical partitions can data be physically stored row based or column based column there in parenthesis on the other hand we have wide column type of no sql databases it's so it's confusing the words column or versus white column um yeah so this is surprisingly confusing cassandra api is a wide column database it is not a columnar store home store yeah columnar stores are are stored in a manner that you can scan by column so they're good for aggregates cassandra is the complete opposite of that um it's it's a row-based store uh like or you know sql api and so on it's it's really geared towards transactions uh but sometimes i've heard it referred to as wide column and then that gets conflated into column now which is something completely different so yeah it's a wide column or column family uh also known uh database yeah and obviously we've implemented it in just the same way interesting uh point though uh there which is um and we don't have it yet but eventually very soon hopefully we'll have integration with synapse link on cassandra that's coming yeah it is coming uh i can't say when but it will come eventually uh and of course as as viewers might know synapse link is is our way of integrating with the analytics during cosmos db which is this real-time sync of transactional data from uh the transactions or to analytics store uh that's kind of drip fed in near real time so you can query that from a from a different engine t sequel or spark and so on we'll implement it in exactly the same way and when that happens that will turn cassandra api effectively into also a um columnar store right with the proper meaning of columnar is that you can actually scan by column and you can um filter on aggregates or it's really um optimized in that way as opposed to transactions both worlds i guess right because you said it was built to be you know high volume fast uh transactional store for the likes of facebook and netflix and now mary ended up with synapse link you're going to get the best in the oltp uh space and all the analytics uh with it being uh also automatically etl to do a culinary store as well right yeah and we think this will be very big because uh as it turns out um spark is a really uh sort of use apache spark is used a lot by the cassandra community to query their data in in cassandra and the reason for that is because it's not really very good with aggregations right or analytic style queries so if you want to do anything like that you kind of have to either pull it out somewhere else or use some some other kind of engine so so the community built a cassandra uh spark connector uh and so a lot of folks are using spark but the problem with that is that exactly the problem that we've solved uh with synapse link is that you you're still queering against your database right so if you're just running a bunch of uh you know it's gonna kill the um uh the throughput available that's us making things easy again yeah darn you darn cosmos guys yeah all right what do you got here let's see what uh this is your change feed yeah so this is this is a sample and this is the i think this is probably the sample that you that you linked before let me just show it here anyway um it's this is i mean this this is really hot off the press let me just say uh that right now so uh even we've even got a little warning down here to say uh um in this feed ranges is in private previews so please reach out if you want to have that enabled on your account you can have a look at this this sample and what this sample is doing is kind of putting an effect into effect what we were showing uh before so there's actually a lot of stuff in here so i'll talk about it although i'll just pick the the main points which is it will create a a table within your key space uh called change feed page state so all that will do is just uh persist um the paging state for each for each feed range and then the first time it runs it will pick up all of the current feed ranges available in that table uh and then it will um based on what you specify in your in your concurrency settings so i've got a concurrency of 10 here um if you if you can specify less than the number of feed ranges you have if you do then it will just divide all of the feed ranges among your threads and then each thread will just kind of execute on around robin as it were but you can't have more concurrency than you have feed ranges so that the number of feed ranges available is the maximum concurrency that you'll get here yeah um so let me just run this thing quickly and see see uh what happens um so so that's that's gonna run here um it's actually not doing very much it's just you know when it when it sees a record it will just i process that and yeah it's saying there rose available zero and you can see i've got um here zero one two so i've got three threads here uh if i was to do uh maybe let's try this didn't try this earlier so i've not prepared myself let's take a risk um if i do a little update on on my table that i'm um kind of listening to on here i should be able to hopefully see the update so let me just do a little update on a table uh you can hide that tray if you want that's good yeah and do i see that update did i see it or did i did i miss it already yeah there you go there it is so yeah anyway uh there's there's there's three threads uh running here so you can see there i specified ten but i i've only got three and what does that mean well that that must mean that i only have um in my project yeah i've only got three um feed ranges um now what i didn't show before is in this this sample here um i think maybe i did show it before i didn't really talk about it much so it's it's fetching the current feeder ranges and what it will do is execute this little query here and this is what we've exposed effectively so all of that that you've just seen is just you know internal management inside this sample but this is what we've exposed that this is a virtual table that you can have access to uh and this will um yeah return back the the feed ranges that are available in your table very tricky so your your filter predicate is actually the from if the front is this virtual thing and then you basically are selling it use where use the where clause for the key space name and the table exactly yeah very tricky i like this yeah and then uh so we get that back um and if i was i can go and query this here uh as well now i see that when i query i actually have have four feeder ranges here so so you can see that it's running with the uh set of feed ranges that i had before absolutely fine and you know but you'll have to trust me there's processing all of the data uh that is available uh with just the previous feed ranges but i've obviously you know the either the data's grown or i've scaled up in the meantime uh well i know what i did i scaled up okay so i scaled up to enough uh request units uh uh that it will give me four physical partitions so if i was to just stop this thing here uh and then delete my um page state uh kind of maintenance table here and then if i was to start this up again it should then pick up whatever feed ranges are available at that time um so let me just i'm sure that delete eventually so let me just start this thing again here and what we should see hopefully is well yeah four four uh four threads right yeah and there we go so zero one two three that's four in total yeah it should have picked up um it should have created a new version of that table um with the four feed ranges in here got it yeah so that's how that's working now obviously um uh you know it's not it's not automatically picking up the new the new feed ranges uh but to be honest if you have a really large l let's say you have a really large uh you know um container or table um and you you know you have uh you you set it at a certain uh uh level a certain point in time with 10 or 20 or whatever feed ranges that's gonna that's going to be a lot more scale than you'd have had previously um so you know if you're dealing with uh hundreds or even thousands of partitions we certainly have customers who who are doing very very big volumes with this thing uh you you could you know change your the concurrency on uh processing change feed by 60 or 100x um straight away over time we'll we'll develop this uh so that we implement um splitting and merging of of um continuation tokens within the feed ranges so that we can then automatically increase uh um the number of feed ranges that we implement on the client side um or that you can implement on the client side so we'll expose um like something like a system function at the back end that will allow you uh to do that is that so that's really interesting like today if you wanted to do something like an auto scale for uh change feed you would need to use the change feed estimator to figure out what is your latency in terms of picking up changes and then you scale it up and down based upon how far behind you're falling you're saying you're going to make that even easier though that customers wouldn't need to do they wouldn't need to implement that well no we're we're going to um we're going to make it equivalent to what sql api does now in its implementation on on you know change feed processor uh within functions and how that's integrated that uses the estimator automatically we're going to bring it in line with that with that capability um there'll still be a little probably a little more hand cranking that you'd have to do as the developer on the client side because obviously we can't control everything there right so so um we'll have to you know play some tricks like we've already done uh with the you know the cql predicate and so on uh but certainly from the capability you'll be able to implement um it in such a way that it will it will scale uh automatically as uh new feed ranges uh you know new subset of uh feed ranges are available and incorporate that into into the the processing um but to be honest even even with this uh it's you know light years ahead of uh of what um you might have in native cassandra for example because there's nothing like this pro programming model to be latest contribution to the uh cassandra ecosystem are they a pretty open friendly ecosystem i'm just curious if uh yeah i think so from what i've seen so far to be honest uh until uh until recently until we've um you know started um um playing around with the with managed instance and and so on uh we haven't had that much interaction with the community because it's always been you know we we've got cassandra api and we you know that we build that's the wire protocol and everything else is you know it's we make sure that we're compliant with what is expected the client side but you know cassandra api is cosmos tv it's all you know um yeah completely separate but but now that we are actually um running a service that is pure apache cassandra at the back end uh and we're developing things that will will you know help that work better and obviously it will also help other versions of cassandra as well then with that where we're sort of um we're only just starting to get into the community in in one sense for the past probably six or twelve months makes sense yeah that makes sense yeah and i know uh when i tackled this episode about cassandra am i but this has been good just talking about all of these other things about the the our own cassandra api yeah rotation as well so i feel like i hijacked it a little bit to talk about because it's okay mi as well definitely um but yeah so the the other feature uh it's actually this is actually in production now at the time i was putting this together i didn't know if we were going to be there but it's actually in production it's not documented yet um but if you know about mongodb api or if you've seen any of the stuff on on that you probably you might know about this already so this is server side retries just to give some context on what this is and why we have needed to have it i should probably set some uh uh groundwork on on request units so um viewers might know uh request units is how cosmos db uh provisions capacity so if it it um uh requesting express capacity in terms of requests per second so if you know the cost of a request and you know your workloads you know how many reads and rights you have and what each one of them costs you can be really deterministic uh about provisioning which is awesome right it's you know if you you can really go to town on profiling what your workload looks like and it's you can be very efficient in how you how you provision that the downside of it is if you don't know your workload on the other side and then you you exceed capacity you get what we call rate limited which can be a little uh frustrating now of course sql api um we build all of the the sdks so um the sdks intrinsically know about rate limiting and they have intelligent uh retry mechanisms and so on all across the board but again it's kind of similar to the last feature in cassandra um we don't we don't have that control right we so the out of box uh drivers don't know anything about uh rate limiting um so what we did uh um or have done is publish things like extensions for java and you know your customer uh user has to implement them uh uh to make sure that it's retrying properly and all of that but obviously you know they'll work differently for different drives and that can be a bit cumbersome and meanwhile if you didn't implement it then you might just hit this rate limiting problem so in the end what we've done is just the same as what what um uh mongodb um has done uh and just to if for those of you you know new to rate limiting or maybe new to cosmos db or uh just to kind of uh illustrate uh the kind of problem that you run into here let me just and again just show a really simple app here um so uh this little app is just all it's doing is just shoving a bunch of records into a table this table happens to be provisioned with only 400 request units so 100 writes per second uh it's going to it's going to overload that table um and to begin with i'm just going to bash it against a a table that doesn't have um server-side retries enabled so let's just see what happens uh when we do that i think you can guess what's gonna happen yeah it'll work for a short while and then yeah straight away you're gonna get these and this is this is what we're contending with uh you know that the driver doesn't know anything despite the fact that it you know cassandra unlike a lot of uh um databases it has client drivers that do have retry policies that you can define but of course it doesn't know the circumstance in which uh um you know you might want to retry for something like rate limiting because it was obviously not designed for that um so what i'm gonna do now um is just switch over to my other um account here that does have ssr enabled and let's see what happens uh when i run this thing again hopefully it will be a little bit better so everything's now yeah there's a bit of latency there obviously because these these requests are being retried at the server side as opposed to on the client uh which means this will work for any driver it doesn't matter what you're uh using but i guess you know what this does is just kind of makes it behave a little bit more like a normal traditional database right like a vm hosted cassandra yeah right classroom it's gonna queue up until the cpu can or the dispatcher can get to them and then execute them and return so we're basically making it feel the same way because we're not running on a vm we're a resource government so you either you get up to what throughput we give you or you provision and no more right so that's uh this is great this yeah this makes a lot of sense uh to do for this and obviously for like too yeah in case anyone wants to know how to enable it's exactly the same as um as uh mongodb you know you've got this property here and you just enable it and that's free right they don't charge for this that's free um uh it doesn't it it won't i should point out it doesn't retry indefinitely obviously so you know like you said we're a resource given database so if we if we were retrying indefinitely then we would you know uh it would be quite difficult to to manage that so we we retry up to 60 seconds worth of retries and then we'll timeout uh but in most cases if you're waiting 60 seconds for a response then it's probably too too slow for you anyway so you know it just kind of takes away the headache and the anxiety of having to to deal with that i mean i guess you're getting a little bit of both worlds then you're getting the the deterministic sort of request unit currency that we've got in cosmos db which is very much uh in line with the kind of past services that provide that kind of thing but also you know you've got this behavior that's a little bit more traditional and something that you're more used to yeah yeah uh what's next oh talk about cassandra mi oh uh you had a link here this elastic scale is this your is this your server-side retry yeah that would be so that's that yeah that you just take a look at that and those are the all of the kind of options around scaling uh in cassandra and it mentions about rate limiting and how to handle that and obviously like i said we haven't documented this yet um but we think that this is going to be a big improvement there'll still be some situations where you might want to have your own retread policy at the client side uh so like for example i don't know um maybe uh you want some some requests to fail and stop uh because you don't want them taking a resource that that other um other systems or other processes are more valuable for or something like that i don't know in which case you would need to have uh different policies i guess for those different clients uh bear in mind server side rewrite applies across the whole account yeah um so you know there might be some scenarios but to be honest in most cases i think most people just want to switch this on i would think if yeah you'd almost uh that most customers immediately turn that on it seems like yeah i think so yeah certainly what we've seen in their manga yeah so let's talk about cassandra m.i yeah finally tokyo this has been a great episode if you've covered so much cool stuff and like i said we've never had we've never talked about cassandra on here yeah and there's a lot of stuff i don't even know so i'm learning a ton and i'm sure our viewers are too so this is great yeah so cassandra mi cassandra managed instance uh i guess it makes sense to to to i've kind of put this as how to choose but i guess it makes sense to talk about where we came from uh and why it was built um so uh i guess i mean this if you if you are somebody who uh you know a company or a customer or user who uses cassandra today uh and you're wanting to move to the cloud any any cloud uh um from on-premises let's say um then you you kind of facing with two choices either you've got something like a cassandra api a cloud native version of cassandra um there's others out there but we were the first so i'm just putting us up there uh um but yeah you've got something like that or you've got lift and shift basically so you're running your own vms in azure or some of the cloud or maybe you'll self-host something like data stacks or something like that so what we've added is this thing called azure managed instance for apache cassandra um and where where this came from the the i one of the the sort of first earliest uh signals that we got for this was that people were talking to to us about xander api and they were saying yeah i want to use it i'm on premises um how can i connect cassandra api to my cluster right how can i do bi-directional replication uh and uh of course the answer to that was well we can't do that because you know that me that would mean that we need to implement uh cassandra's gossip protocol it's internal uh protocol for replicating data between nodes within a within a data center and within a cluster uh and this this ask came up quite often uh surprisingly uh and so what we were seeing was that although customers some customers obviously uh we we have plenty of customers um but there were customers that were wanting to use cassandra api but they're not for whatever reason quite ready to completely dive into it uh or they they kind of wanted to use it as an extension of their existing environment right for for whatever reason so this was interesting and for various other reasons as well we were seeing that there were you know there's a gap between being on premise and using cassandra and being really invested into into the platform because it's very it's a very platform-oriented database right you get into the back-end aspects of it there's a big gap between being in that space and moving to cassandra api even though cassandra api has a you know a wire protocol um you know we've we've made it as compatible as we can and to make lower the friction if you want to move to something like cosmos db there's still this gap uh between the two and what we uh you know like being able to uh deploy a hybrid cluster i guess they say that the um what's the saying um necessity is the birth of innovation is it so you know when you need to do something as things that you have to innovate on and we realized that we were going to need to do this in some scenarios or uh to meet certain customer demand it turns out it's not been done before or at least certainly not for a cassette a cassandra managed service uh or any other type of managed service as far as i can tell this is where you have a managed service but you're allowing it to connect in some way uh to make a kind of heterogeneous environment out of a self-managed version of that database or self-managed component of that database and the managed component so in this case you have a um a cluster that is formed of a data center or many data centers in azure managed instance for cassandra and a self-hosted data center and the two are kind of merged together and connected together via networking and we are sharing um certificates and and and seed nodes that represent uh the core nodes within it within a given data center and that's that's what we've provided uh for this um for this service and that has two things one it answers that initial question that we were getting for customers but it also makes it a lot easier to migrate um it's as it turns out we can't build a better migration tool than cassandra itself because it's replication replication is obviously very seamless and it you know when you have this capability to add data centers into a cluster ring and you inject those resources into a virtual network which is what we do with the managed service that allows you to connect it to anywhere else that you want and effectively you can create a hybrid cluster so that's what we've built in there the uh gossip protocol do you go back it's like so how are the uh does that require some special networking i mean can i do this over public internet or do i need some kind of private network to do this or it doesn't it doesn't require a special networking it runs on a certain port but with managed instance the one thing that we do insist on is that your existing uh data center or cluster uh that you want to configure a hybrid cluster with has to implement um um uh tls uh how to implement certificates uh node to node encryption uh it's referred to um so that's the one thing that we insist on uh but aside from that i mean the networking uh um to to do this for uh on premise would be typically something like express route okay and then you know virtual networks uh appeared between any additional data centers that you have in managed instance uh or just using express route to to do you know you make a circuit between uh um certainly in the cloud or on primary as well either way it is fine as long as there's a clear route through back and forth between the different resources then it's just a matter of you know sharing uh certificates and defining the seed nodes in each in each environment um so we provide the mechanism of doing that as i said there's no other uh managed service for cassandra that's allowing uh uh folks to do this um so we'll see we'll see how that goes uh cool right we're being a first in that regard i mean uh i can see how this could make uh migration for cassandra existing cassandra users that are managing clusters of cassandra uh so much easier well yeah i mean i i you know i posted it the other day um it is the most i mean technically it is the most seamless way of migrating uh an on-premise let's say apache cassandra cluster to a managed service in the cloud i can't see any more seamless way than using cassandra replication to do it um so yeah um there were the caveats of it i mean we'll come to that later on but you have to be running something that's compatible with what we support uh but aside from that you know if you are um then it's really super straightforward to get your you know get your environment up and running in the cloud or even just have it run it as a hybrid cluster for a while and that will be seamless as well and then once you're happy you know you you just decommission what you have for running on premises there now when you do our cassandra mi i'm guessing you can create multiple nodes so you can have that fault tolerance uh as well is that correct yeah so you have you you need to have at least three nodes in in any given data center for so for a quorum and typically we say you should have an odd number of nodes for the same reasons that we do that in cosmos db okay so well i mean i guess what we envision here uh and certainly based on the evidence of what we've seen uh from talking to customers is that most customers who are using um cassandra on premises they might want to use cassandra api or something like cassandra api but they're probably going to want to do this right they're going to want to lift and shift uh and it's going to be very interesting uh to be able to lift and shift so seamlessly into something that's managed and does things uh you know a large um percentage of things for them and then we see we see maybe later on they might want to take advantage of cassandra api and maybe they'll even migrate fully into cassandra api but they don't have to they could just stay here or they might even stay here it's really up to uh the use of the customer uh to decide what they want to do we haven't we haven't built this uh with a view that everybody's gonna eventually decide to use cassandra api or cosmos db it's really you know it's a standalone uh product obviously we got the experience of how to run distributed databases so and we've got the experience of talking to cassandra customers who want to get into the cloud they want it to be managed and easier but we've learned that you know it's um it's not quite so simple as just giving them a wire protocol compatible version of cosmos db like i said it if their starting point is you know they're really invested in cassandra there's quite a big gap between that control that you see there on the left-hand side uh and you know the productivity that you might want to get to with something like cassandra api and from the perspective of some folks uh depending on their workloads they might be giving up too much in that case so really you know we're gonna just see how this thing goes uh if uh you know if it becomes popular then then you know becomes popular if it acts as a stepping stone then that's fine uh as well and like all of these things that i've listed here i'm not going to read them all out but basically they kind of collapsed down into this uh this is what scott hanselman this is the kind of uh analogy that he gave last time i was on uh i really liked it because it really just describes the the situation right if you're if you're talking about um the platform itself in terms of running the database platform um cassandra api is really liking an uber it's doing everything right we're running the clusters um we are um maintaining everything we're doing patching we're we're doing all of that you you all you're concerned about is creating key spaces and tables and schema and the application side right uh other end of the spectrum if if you obviously if you're self-hosting you're going through the gears yourself and then published instance is like an auto shift right you're still in control you're still driving if you want to think of it that way and actually where we're going with this is is going to be more like a semi-auto shift right so we're going to give as much control as possible so you can shift through all of the gears uh in other words we will do a bunch of automation so we'll automate deployments automate scaling with scaling down of nodes automate um uh operating system patching security patching for cassandra um running reaper to uh um do auto repairs on different nodes and so on and various other things but you'll be able to override all of it um potentially like you can in an in a semi-auto shift right you do the paddles on the on the steering wheel uh we won't give everything uh you know you can think of it like uh the clutch i guess we won't let you with that we'll still be maintaining um the operating system and that will be locked down but from the cassandra level upwards the intention is eventually certainly by by the time the services ga to give you the complete control over everything if you want to override it you don't see the need to override it you can let us do it and it's really the choices up to the customer yeah that's cool uh [Music] i mean that's um you know if you're sick of managing clusters then why not go with just uh letting us do everything right or just touch the things you absolutely need to touch and yeah so i love it and scott scott does a great job yeah yeah it's it's here yeah so i'm going to continue to steal that yeah um yeah so i mean i just talked very quickly about how to migrate um so what's happening here okay yeah so so i like i already said before um we've now you know um got a um by far the easiest most seamless option for migrating if you're running on premise with cassandra and you want to get into the cloud with some kind of managed service it doesn't get more seamless and um more straightforward than this really if you happen to be running um a version of cassandra that we support version 311 or higher or close to 311 um even if you're running data stacks and and the the underlying cassandra version uh matches or is close enough it's going to be possible at least uh for you to do this and then once it's all set up it's really you know as i said you know there's no there's no better tool than the database itself uh for replicating itself to other environments if that's not possible uh for whatever reason maybe the version doesn't match or you maybe you're running um some other y protocol uh compatible cloud surface other than cassandra api i don't want to mention on your names um then you could use uh the dual rights approach so i've we've documented this already we did a video uh on azure friday um you want to go and check that out this this process uh we've we've the approach to this has been around for a while people have been doing dual rights to achieve migration for a while especially if there's not much data historic data or any historic data to move obviously it's straightforward to just get your app to write to two places uh but we've made this a little bit more straightforward by just lowering the friction on the app side so we created this dual write proxy that you install on your source cluster uh and it runs on a different port uh and you install it on all the nodes and then and then um you just connect your application point it to a different port and the proxy will forward all of your rights uh to to to your target so it just reduces the friction having to make changes all over the place on the app side um and so that's all documented and you uh definitely uh check that out to be totally transparent in theory it works for any uh form of cassandra including cassandra api if that's your source so maybe maybe i should have lowballed that really but anyway we're in the open source community now so you have to you know if you want to go from uber to automatically i guess you can well yeah or manual it's a it's a manual it's entirely up to you and then yeah this offline options so we we recommend spark it's especially with databricks and azure or some service like that um you know it's very straightforward um for offline migrations are very scalable and we've documented the approaches to that as well um so i wanted to do a little bit of a blue peter style demo do you i don't know if you you do you know uh do you know what blue pizza is no i guess it's not very i guess it's not very famous over there it's it's a it's a british tv show uh which is famous for people it's a it's a children's tv show where they do like you know demos of things or they like do uh um arts and crafts and things and they'd um they were famous for always saying and here's one i made earlier so that's what i mean you like tv shows i know i don't know why my son seems to love the ones coming out of australia so yeah peppa pig uh oh his new favorite is bluey uh out of australia i don't know what it is with my side and australian tv shows but i haven't seen that one so yeah check it out check it out it's still on okay going it's been there since the 50s oh okay yeah um so anyway here's one i made earlier here's a hybrid cluster that i made earlier um and uh what this is so as you might expect i mean obviously it would be it would have been a bit difficult for me to do this on premises and and show it easily so i've just installed um a vanilla cassandra uh cluster on vms in azure in a particular region so you can see i've installed this in west u.s too uh and i have uh three nodes installed on vms and i also have uh cassandra managed instance cluster resource i'll explain a little bit more about that in a minute but for now i just want to run this uh node status come on this is from my uh um uh source self-hosted cluster known you can see there i've got a data center here that's my self-hosted data center and down here is my manage instance data center and some typical thing that you might want to do if you've if you set this up uh is that you might want to use uh your uh managed instance uh cluster for adding additional capacity right and the reason that you might want to do that is because normally on cassandra uh if you wanted to increase the number of nodes in your days and you'd have to be going and you know grabbing some more vms and installing the software and making sure that um you run the right no tool commands to make sure uh that note comes up properly within the ring and tokens are distributed and so on uh but in this service uh it's just a cli command right so i'm just gonna run this command and i'm gonna scale up the number of nodes here uh from three to five um not found live demos what's going on here oh i'm in the wrong window that's why so this is yeah i'm actually connected to my self-hosted uh node there says no okay i think but an a z command is not going to work there so let me just place this in here uh okay so i'm just scaling up uh my managed instance uh data center from three nodes to fire so while that's going on let me just go back in here and let's take a look around here so like i was saying before um yeah i've got the i've got my vms here but i've also got this um let me refresh this here uh there we go yeah so my cluster resources here and that so what happened the way you do this is the hybrid cluster i have my existing cassandra cluster uh when i want to set up a hybrid cluster i create a managed instance cluster with the same name uh and i provide that cluster with uh the um tls certificates of my um um nodes that should have had node to node encryption enabled and also the seed nodes as well so this the nodes within that data center and then i create a data center uh and then i do the same thing in reverse i give my certificates to my existing add that to the truster of my existing uh environment and i also add the seed nodes there and that's it as long as you have the networking setup that's all you have to do and then you've got a hybrid cluster uh and then what that ends up with is uh you don't see the resources in here because we manage these you can see them they're visible inside the v-net so we inject uh the nicks of the um of the nodes that we provision uh into into the v-net uh but you don't see the resources in the kind of traditional way because they're hidden they're in a in a different tenet what we're providing the the interface cards so that they can communicate with anything inside your uh virtual network um so that's kind of how that that works right now the display this still the service is still in in public preview so this will uh what you see in the cluster it's still pretty fairly uh basic this is going to improve there's going to be more uh information that you can see about your cluster and drill down into nodes and things like that nodes and stuff yeah yeah that's always you know wonders like where i see nodes um yeah so the way the only place you can see them is in the v-net that you can view the the injected mix but part of that and that's the data center name that you see there you can see everything that you need from cli commands right now very cli because you see a line do you get on there right i think yes people that um i don't know i'm presuming most people that manage clusters for cassandra are probably used to dealing with things in a shell yeah exactly so i'm going to do that right now so let's let's uh um while this was running um what's going on here okay so this was this is oh okay this is still running that's okay i can i can do this in a different uh let me fire up a different uh window here um and while that is uh scaling up i mean there's a bunch of commands here but you just mentioned uh do a get so let's let's try getting the node status of the cluster um and what i was so what's some something to point out without what i did with my uh uh sort of initial uh note here let me go back to the original command that i was running uh the no tool uh command to get status is going to tell me everything about the cluster but when i run this uh from cli it will only give me information about the data sensors that i have provisioned in managed instance um so to get the full status of your on-premise uh resources you'd have to run no tool from your on-premise environment but so let me just run this and see what what that comes back with this is where i'd normally be editing to make it run faster but i have to sit here and wait for it to yeah it's uh like as the uh as the control plane pm i'm used to my demos running yeah painfully slow because because the arm itself i mean just doing these types of operations always takes time so there's no no latency sla on these things yeah i guess what we should be seeing right now at some point at least uh is no we're not seeing it yet i mean this can take a variable amount of time uh so i mean i we should at least see that scale sets are kind of spinning up uh yeah so yeah so you can see some more this is the one so don't be alarmed when you see more nodes that you expected this is the way scale sets works it has to provision a number of different uh addresses before it decides on the ones that you wanted as crazy as that sounds uh hey you know we're we're using scale sets in this service so that's what that's doing um but yeah eventually uh this is going to add uh nodes to the cluster and then add nodes to the ring and then we should be able to see this in here um eventually so maybe we should either wait that's pretty simple just one line cli add update the number of nodes and hit enter i mean yeah pretty straightforward so this is well this is my node status command so this is telling me uh what nodes are available in the cluster and right now it's saying there's only there's only three so that is obviously that scale-up command it's not finished doing stuff yet so it's telling me these are the nodes that are available and running normally um so for whatever reason this is running slower than i was hoping so maybe we can uh uh yeah that's uh that's live demos for you yeah yeah yeah but it does work anyway just not for me today very cool okay yeah so that was it maria i just you know a little uh here's one i made earlier um like i said i've you know we've documented all of this how to set up a hybrid cluster how to set up multi-region uh with your um with your managed instance um how to run cli commands and there's all the kind of things that you would expect in there and then at the migration stuff that i talked about as well uh with the dual write proxy is all here uh and the spark offline approach as well um so yeah there's a bunch of stuff to investigate there it's it's developing it's still in preview um but we've seen a lot of interest uh from folks as we expected because of the you know the feedback that we were getting uh while we just had cassandra api um so i'm pretty confident that when when this goes ga we'll we'll uh we'll be busy yeah well this is awesome uh theo i can't believe we're we're like well over an hour into our show here and uh it's just there's so much stuff you guys have going on uh you gotta promise to come back when you uh push more stuff out or get more of these things uh released oh definitely i'll definitely be when this is ga i'll be uh yeah come on back and then and then next year too when i saw that stuff for fy22 on their materialized views and all that other stuff the named indexes i can't remember all that other stuff but you have a lot coming uh next year so uh definitely come on back i want to learn more so yeah awesome cool thanks man yeah well thank you very much uh theo uh good having you on thank you everyone for joining us uh this week i hope uh you enjoyed learning about our cassandra api and also cassandra manages cassandra managed oh no no no let me get it right manage the instance for apache cassandra is that it did i get it right yeah yeah all right i gotta i gotta i gotta stay on brand or marketing's gonna come find me so actually actually managed instance for apache cassandra oh okay oh yeah i forgot the azure part so all right well thanks for thanks for correcting me oh there we go it finally finally finished did we did it have we got time to see if actually i want a successful demo well it's finished adding uh so it does this asynchrony so it's finished adding uh the nodes uh but it will take some more time to add those those nodes to the ring so uh i'm guessing they have two there we go finally one of them is joining you see that was joining ah okay okay so i'm getting there to provision the resource the data has to backfill and then it has to complete the the final step in joining the ring there so that's about it yeah exactly yeah so believe me this is a lot quicker than doing it yourself yeah way easier too yeah oh there you go now you've got four notes so there we go cool anyway i won't wait for the rest oh that's cool thank you very much uh and thank you everyone for joining us this week uh just some announcements kind of rounded out um last night we had our very first uh global azure cosmos db user group uh meetup uh we had i think a little bit of technical issues there but uh if you missed it and want to check it out we've got a recording uh just head to uh the link here and then look for our meetup and then you can find the link to the recording uh in the discussion uh down in the comments uh for our meetup last night uh we don't have our next months scheduled uh yet but we're gonna get there here pretty soon uh so keep coming back and joining us and join our user group uh up around meet up and then uh come hang out and watch our monthly user groups uh i'm gonna try to do giveaways for our little neoprene uh cosmos tv uh um drink coaster and a little cosmos db sticker uh as well we're trying to successfully do that uh next time for our our user group uh also coming uh next week uh it's almost here azure serverlessconf uh september 29th through the 30th we'll have three three hour live streams starting uh 9 a.m on thursday uh the 29th is that thursday i can't remember if it's thursday i think it's thursday the 29th whatever it is the 29th 9 a.m uh we'll kick it off uh and then later on uh that evening uh at least evening pacific time or evening u.s time uh afternoon australia time we'll have uh the apex stream and then after that we'll have the emea's live stream so come check that out uh we'll have sessions live for you unique sessions uh for each of the three live streams uh wherever you are in the world uh take some time and come and watch uh community and see what they're doing uh building serverless event applications using azure serverless technologies so come check that out we'll have lots of live our on-demand sessions available too uh got any show ideas drop us a line at your cosmos tv on twitter uh always interested to see uh if anybody has any interesting show ideas uh next week we're gonna have uh andrew lubeck uh and he's we're gonna finish the conversation we started back on episode 20 uh which was gosh about six weeks ago now because this is episode 26 uh and he's going to go into details on the things we kind of touched on like like our gateway and how that works and how our query engine works and how kind of the quorum mechanics works for our replica sets both locally and then also talk about replication as well our compute layer indexing more change feed all kinds of stuff so uh we'll have him available next week and come and check that out uh ask lots of questions uh it's always great to have him on i think he's basically a founding member founding member of the team uh nearly from a pm perspective uh so he knows he knows lots of stuff so you can ask all your all your questions so that's it for us this week uh theo thank you very much sir it's great having you on pleasure and uh okay folks thank you we'll see you next week [Music] [Music] foreign

Channel: Azure Cosmos DB

Views: 263

Keywords:

Id: QR45oXmmY7o

Length: 79min 7sec (4747 seconds)

Published: Fri Sep 24 2021