Some of you may have heard about the new A.I.
chatbot called ChatGPT, which was launched in November 2022.
It's generating a lot of
excitement and consternation because of its ability to answer questions, troubleshoot
computer code, and write college essays that are surprisingly good.
There are already dozens
of YouTube videos touting its various uses, including aiding in scientific research and
writing.
Hi. I'm Karen McKee and I thought I would test out ChatGPT and see what it can and
cannot do with respect to scientific writing.
You can read about ChatGPT at openai.com.
And
you can try out ChatGPT at chat.openai.com.
It s free to use at the moment and millions
of people have already tested it.
Registration requires an email address.
This is the simple interface where you can
check out some examples of the output and read about its capabilities and limitations.
To the left are listed all your chats.
Below are buttons to clear all conversations.
Select light or dark mode.
Link to Discord, which is another chat app by open ai.
Read updates
and frequently asked questions.
And log out.
And here is where you can
type in your questions.
I started with a question related to
my own research. What is the effect of sea level rise on coastal wetlands?
And in seconds it spits out an answer.
Sea level rise can have a significant impact on
coastal wetlands.
As sea levels rise, saltwater can intrude into freshwater wetlands, causing a
process called saltwater intrusion.
This can lead to the loss of freshwater wetlands, as the
saltwater can make the environment inhospitable for many of the plants and animals that typically
live there.
It goes on to talk about flooding impacts and loss of ecosystem services.
And ends
with a caveat that the effects of sea level rise will not be the same in all places.
This is
a surprisingly good answer to the question.
I then tried to trip it up by asking what is
the effect of sea level rise on beagles.
And it provided a sensible answer, pointing
out that sea level rise would not likely have a direct effect on beagles as they are not
normally found in coastal wetlands.
But then talks about indirect effects via storms and
flooding which could affect pets such as beagles and their owners.
Pretty impressive.
I next
tried a variation on the sea level rise question, asking about responses of specific types of
coastal wetlands.
And again, I got another pretty good answer with three suggested mechanisms
whereby coastal wetlands might accommodate sea level rise.
The chatbot nicely bookends this
information by initially pointing out that these wetlands can accommodate small rates of sea
level rise
and ends by saying that they may be negatively impacted by more rapid or extreme
changes in sea level and that changes in sediment and water flows may make it more difficult for
these ecosystems to survive sea level rise.
Although I would word some things differently
and modify the part about how vegetation aids in adjustment to sea level rise, in
general, this response is reasonable.
But, there s something missing.
Citations
for the sources of this information.
When I asked for sources, I got this response.
Which is not a good sign.
Scientific papers must reference all sources of information.
They need to be relevant and up-to-date.
But let s see if this chatbot can help
with coming up with a title for a paper.
Here s a paper reporting on the effect of sea
level rise on salt marshes and mangroves.
I ll copy the abstract and paste it into ChatGPT.
And
ask it to suggest 5 titles for this abstract.
I paste in that abstract and hit return.
It s listed five titles, some not bad. Some
not so great.
But they represent five very different variations on a theme,
which would get
you thinking about some possible titles and lead to one that best represents the paper.Let's see
if it can get us started on the introduction. Write an introduction to a
paper with this abstract.
Coastal wetlands, such as mangroves and salt
marshes, play a vital role in protecting coastal areas from the impacts of sea-level rise.
These ecosystems are able to build soil surfaces vertically through the accumulation of inorganic
sediment and organic matter, helping to keep the wetland surface above water.
However, at climatic
boundaries where mangroves are expanding and replacing salt marshes, the wetland s capacity
to respond to sea level rise may change.
Okay. That's not bad. It's taken the information
in the abstract and the previous response to our question about sea level rise effects on coastal
wetlands and changed the wording so that it doesn't sound too repetitive of the abstract. But there
needs to be more information about mangroves and salt marshes and how they may differ in response
to rising sea level. So, let's ask can you expand upon the characteristics of mangroves versus
salt marshes in response to sea level rise?
Okay. Both live in areas subject to tidal
flooding but have some differences.
One key difference is that mangroves
have specialized roots allowing them to grow upwards whereas salt marshes do not. Well, that's not accurate. Both accumulate root matter contributing to soil volume and
upward expansion of the soil surface.
Next is salinity tolerance. No. both mangroves and
salt marshes can grow in full strength seawater.
Some mangrove species are better at tolerating
hypersaline conditions but sea-level rise is not necessarily going to cause hypersalinity.
So this answer shows that ChatGPT is relying on some information that is not exactly accurate or
at least requires a more nuanced explanation.
Let s see what happens when I question
the accuracy of this response.
I say that this information is not entirely
accurate and briefly explain why.
And the response is to accept the correction,
even though I provided no evidence to support my criticism. And it also modified the salt
tolerance statements. Well, I think this exchange shows that you must be cautious
in accepting what the chatbot gives you.
I wonder how the chatbot will do with writing
a discussion when given a set of results?
I'll go back to the published paper and copy
the results section which is fairly short. And ask it to write a discussion based
on these results.
And paste those in.
As you can see, it's basically restated
the results using slightly different wording and without any interpretation
or comparison with previous work.
What happens if I ask for a
comparison with previous work?
It replies that this cannot be done
without more specific information.
Okay, let s be more specific.
Still not able to make a comparison.
So, this exchange shows a serious limitation because
of lack of access to such external sources.
I would need to provide the details
of the Perry and Mendelssohn study.
Which is what we did in our paper's discussion
and concluded that our results were similar. If you are a novice at writing, ChatGPT may
be able to help you polish your paper.
Let s ask it to improve a passage using active voice.
Three mangrove species were subjected to treatment with freshwater or saltwater.
Leaf number was
negatively affected by salinity, but some species were more affected than others.
This result was
found in both greenhouse and field experiments.
And we get, We subjected three mangrove species to
treatment with freshwater or saltwater and found that salinity negatively affected leaf number.
Some species were more affected than others, and this result was consistent across
both greenhouse and field experiments.
That flows better and is about the same
number of words.
Let s try another one.
Can you improve this passage.
The
nitrogen concentration was high in January, a bit lower in February, even lower in March,
and reached the lowest point in mid-July, after which the concentration increased
from August to December.
And we get, the nitrogen concentration fluctuated
throughout the year, with a high in January, a decrease in February, and a further decrease
in March.
The concentration reached its lowest point in mid-July, before increasing from august
to December.
That s better but not what I was looking for. I'll be more specific. Can you be more concise with fewer monthly details? The nitrogen concentration fluctuated throughout
the year, reaching a high in January and a low in mid-July. It then increased from
August to December. That's better. Let's try one more and ask it to
make this paragraph more cohesive. You can pause the video to
read it and the response.
The paragraph now flows better and is not as
choppy as the original.
I could continue with more examples, but I think you get the idea.
If
you have a clunky passage, Chat GPT may be able to reword it for you.
This may be especially helpful
for those trying to write in a second language.
What else might ChatGPT do? Can you suggest 3
directions for future research on this topic? And it suggests studies into the mechanisms
underlying differences in elevation and other factors. Effect of differences on health and
functioning of these ecosystems. And management interventions.
All reasonable ideas that might
be mentioned in the conclusions section of a paper reporting these results.
What about help with
social media and making your work more visible to the scientific community and beyond?
Can
you suggest 3 tweets based on this research?
Well those are pretty good and
it even suggests some hashtags.
How about can you explain this
research to a 12 year old?
And we get.
Sure. So, this research was all
about how well different types of wetland plants, like mangroves and salt marsh grasses, can adapt
to rising sea levels.
When the sea level goes up, it can be hard for these plants to survive
because they can get flooded by the water.
So, the researchers wanted to see how well different
types of wetland plants can cope with this kind of change.
To do this, they set up a bunch
of plots with different types of plants in them and then measured how well the plants
were doing over time.
This wording might come in handy if you are interviewed by
the news media or members of Congress.
Especially if you struggle to put your research
into everyday language.
Okay. Let me wrap up by going over what I see as some pluses and minuses
to using ChatGPT to write a scientific paper.
On the positive side, ChatGPT can help you
rewrite problematic passages in your manuscript, summarize results, or make suggestions for titles
of your paper or future research directions.
