Meet JetBrains DataSpell – The IDE for Professional Data Scientists

[Music] hey everybody this is nothing from jetbrains again i didn't think i'd be making a video like this today but here we are um pycharm is well the pycharm team has been working on a new ide called data spell and what data spell does is that it answers the question of what is what are we doing for data scientists out there right so data spell is a new ide that is designed for professional data scientists at the core of that experience is jupiter notebooks we've worked really hard to make that jupiter notebook experience as clean as possible so without much further ado i wanted to give a quick overview of what we're doing in data spell this is the first uh really public eap we've had eaps before but this is the important one and we're going to have a really soon after so this is an overview and we hope you like it and we'd really appreciate some feedback [Music] jetbrains data spill is a brand new ide that is designed for professional data let's gain an overview of what it can do for us this is what you get when you first launch data spell you're asked for what environment you want in this case we have conda and i am going to go ahead and use conda to launch our environment and workspace the first thing that i'm going to do is attach a directory directly to this workspace i'm going to click on the attach workspace button and then pick a directory where i have all my sample files data spell has support for python files jupyter notebooks as well as csv files in this case i'm just going to open up the notebook file and run a few cells just to demonstrate what data spell can do we've really gone back to the drawing board on our notebook support so that it's more fluid and just works better to start off anywhere you see text is a markdown block for code blocks you have both code as well as the output in other words it works exactly the way you'd expect a jupiter notebook to work but with all the awesome pycharm features as well for example here when we select read csv we also get the auto import for pandas we also see code completion for files in read csv now i'm going to do another pycharm thing which is to use extract variable and call that data and that brings us to our next data spell feature which is an awesome data frame rendering engine so once this is loaded we'll be able to see the data frame and we can scroll down it and sideways all the good stuff that you'd expect but to add to that you can open up this data frame in a new tab as well now going back to the notebook itself we can keep running the different cells by pressing shift enter you'll notice that all of this is running in the background it's managed the jupyter notebook is run by data spell and now when we get to something that generates images we can also see those images being rendered inside of the notebook data spell also supports the ability to debug within a notebook so in this case i'm going to hit the debug button after having set a breakpoint and i'm going to see the debug tool window pop up now you have your regular step overs and step into's but you also have the ability to view variables as series which is pretty cool you also have the ability to evaluate any expression and add that as a watch and you of course get code completion for it and you can also view it as a data frame or a series or an array whatever the variable works with okay so that's enough of notebooks what about python files well let's open up titanic.pi you can run the entire file if you want to but if you take a look you will see cell dividers these selva dividers allow you to work with parts of your code at a time and you can see that they have a special icon next to them as well so we're just going to execute that particular block and we're going to see it show up in the console you can also run a single line by clicking this button you can also run only what you select so here if i just select data and click it again i can see that only the data frame is showing up and yes the data frame is rendered very nicely inside of the console as well you can also see images rendered in the console as well as see the variable viewer on the right hand side of the console well that's all folks thank you so much for watching we also want to thank everybody who helped us make data spell possible from the early users to the bug reporters thank you so much this is still very much a work in progress it is our first eap our very first public eap and we appreciate all the feedback that we can get so keep that feedback coming [Music] you

Channel: JetBrainsTV

Views: 13,172

Rating: 4.9603958 out of 5

Keywords: JetBrains, software development, developer tools, programming, developer, data science, jupyter, jupyter notebooks, Exploratory Data Analysis, machine learning

Id: YRC4ZJKS4vk

Length: 5min 34sec (334 seconds)

Published: Tue Sep 07 2021

Reddit Comments

[u/MrBurritoQuest]

Seems to have similar functionality to VSCode with jupyter extension, is there anything new here that I’m missing?

[u/xDragod]

Looks awesome. Love the ability to open a dataframe in a new tab. Wish I could convince my company to get us licenses.

[u/krypt3c]

Really like how the notebook layouts look like a standard notebook with markdown and all now. Any plans to have a table of contents feature like jupyter lab/notebook? That remains the killer feature that has always kept me there.

[u/wioym]

Hey everyone,

I've been using DS since May and can say that there are some flaws, which are getting fixed with every update, and quite significant advantages.

I am a huge fan of debugging, this is one aspect that was lacking for me in the VS Code (sure you can export, but that's too much of a hustle). I can say that in terms of debugging DS is 100x times better than VS Code, you can set up a break point and watch the magic happen!

Edit: Also ability to see full DataFrame and Numpy array as a table is an amazing thing!

It is much closer to jupyter-notebook than vscode, which with the recent updates decided to go on its own pathway.

[u/Taipan100]

Adding in-cell break points seems like a really valuable feature. As others have said it would be good to see remote execution such that I can work on a docker container.

[u/molivo10]

i really like pycharm but dataspell looks very underwhelming, especially compared to existing alternatives

[u/unplannedmaintenance]

I would really love to see better support for remote Jupyter servers, especially variable inspection.

[u/set92]

Don't see the coolness of this, as others have pointed out, there isn't much more than there are in pycharm scientific, or in Jupyter.

So, okay, they have an interface that behaves properly, and probably going to charge quite a bit for it, but doesn't have git nor docker integration, they didn't show if it was possible to sync with remote work envs like AWS or how they handle big data, we can send the processing to some spark in databricks or some place?

[u/YouAreFine]

Hey, the notebook layout looks nice! A couple of questions:

1. Will these features for notebooks be available in PyCharm as well?
2. Any plans to improve the "remote Jupyter server" workflow? I really like VSCode's "reopen in a container" feature, since keeping ML environments in containers is so much cleaner.