44 | Importing and Exporting |
Everything we’ve done so far in this book has been done entirely within the Wolfram Language and the Wolfram Knowledgebase. But sometimes you need to get things from the outside. Needless to say, they often won’t be as clean and organized as what we’re used to inside the Wolfram Language—and they may change without warning.
As a first example, let’s import the text from the front page of the United Nations website. We can do this using the function Import.
Import a text version of the front page of the UN website (it might be different now):
In[1]:= |
Out[1]= |
The result is a string, possibly with some blank lines. Let’s start by splitting the string at newlines.
In[2]:= |
Out[2]= |
In[3]:= |
Out[3]= |
Import lets you import a wide variety of different elements. "Hyperlinks" gets hyperlinks that appear on a webpage; "Images" gets images.
Get a list of the hyperlinks on the front of the UN website:
In[4]:= |
Out[4]= |
In[5]:= |
Out[5]= |
As a more sophisticated example, here’s a graph of the hyperlinks in part of my website. To keep it manageable, I’ve taken just the first 5 hyperlinks at each level, and gone only 3 levels.
In[6]:= |
Out[6]= |
The Wolfram Language can import hundreds of formats—including spreadsheets, images, sounds, geometry, databases, log files and more. Import will automatically look at the file extension (.png, .xls, etc.) to determine what to do.
In[7]:= |
Out[7]= |
In[8]:= |
Out[8]= |
As well as importing from the web, Import can also import from your own files, stored in your computer system or in the Wolfram Cloud.
The Wolfram Language lets you not only deal with webpages and files, but also with services or APIs. For example, SocialMediaData lets you get data from social media services—at least once you’ve authorized them to send the data.
Find the network of my Facebook friends who give access to their connection data:
In[9]:= |
Out[9]= |
In[10]:= |
Out[10]= |
Request image thumbnails:
In[11]:= |
Out[11]= |
They’re recognized as different kinds of birds:
In[12]:= |
Out[12]= |
An important source of external data for the Wolfram Language is the Wolfram Data Repository. The data in this repository comes from many places—but it’s all been set up to be easy to work with in the Wolfram Language.
You can find out what’s available by browsing the Wolfram Data Repository.
Once you've found something you want, just use ResourceData["name"] to get it into the Wolfram Language.
In[13]:= |
Out[13]= |
In addition to getting things into the Wolfram Language, you can also send things out. For example, SendMail sends email from the Wolfram Language.
In[14]:= |
In[15]:= |
If you want to interact with external programs and services, you’ll often have to export things from the Wolfram Language.
In[16]:= |
Out[16]= |
You can also export to local files using Export.
In[17]:= |
Out[17]= |
Here’s part of the resulting file:
Import the contents of the file back into the Wolfram Language:
In[18]:= |
Out[18]= |
The Wolfram Language can import and export hundreds of formats, of many different kinds.
In[19]:= |
Out[19]= |
Creating 3D geometry in a form suitable for printing can be quite complicated. The function Printout3D does all the steps automatically—and it can also send the final geometry to a 3D printing service (or to your own 3D printer, if you have one).
Make a random clump of spheres:
In[20]:= |
Out[20]= |
In[21]:= |
Out[21]= |
Import[loc] | import from an external location | |
SocialMediaData[...] | get data from social media networks | |
WebImageSearch["keyword"] | do an image search on the web | |
ResourceData["name"] | get data from the Wolfram Data Repository | |
SendMail[expr] | send email | |
CloudExport[expr,format] | export in a certain format to the cloud | |
Export[file,expr] | export to a file | |
Printout3D[source,"service"] | send source to a 3D printer service |
9 Exercises Available
Get Started »
44.5Use ImageInstanceQ to find pictures on https://en.wikipedia.org/wiki/Ostrich that are of birds. »
44.6Use TextCases with "Country" to find instances of country names on http://nato.int, and make a word cloud of them. »
44.8Send yourself mail with a map of your current location. »
44.9Send yourself mail with an icon of the current phase of the moon. »
Why do I get different results when I run the website examples?
Because the websites have changed! You’ll get whatever they have on them now.
Why doesn’t Import retrieve elements that I see when I visit a webpage in a browser?
Probably because they’re not directly present in the HTML source of the webpage, which is what Import looks at. They’re probably being dynamically added using JavaScript.
Yes. Use the upload button in the cloud system to upload the file into your cloud file system, then
use Import.
use Import.
What formats can Import handle?
Where does Export put the files it creates?
If you don’t give a directory in the file name you specify, it’ll go in your current directory. You can open it as your operating system would using SystemOpen, and you can delete it with DeleteFile.
What is an API?
An application program interface. It’s an interface that a program exposes to other programs, rather than to humans. The Wolfram Language has several APIs, and lets you create your own using APIFunction.
How do I authorize a connection to an external account of mine?
When you use SocialMediaData or ServiceConnect, you’ll typically be prompted to authorize the Wolfram Connection app for that particular service.
- ImportString lets you “import” from a string rather than from an external file or URL. ExportString “exports” to a string.
- SendMail uses either mail server preferences you set up, or a proxy in the Wolfram Cloud.
- The Wolfram Language supports many external services. Typically it uses mechanisms like OAuth to authenticate them.
- Another way to get (and send) data is through direct connection from your computer to a sensor, Arduino, etc. The Wolfram Language has a whole framework for dealing with such things, including functions such as DeviceReadTimeSeries.
- If you’re running everything locally on your computer, you can have the Wolfram Language start external programs, and exchange data with them, for example using RunProcess. In simple cases, you’ll often be able to just pipe data straight in from a program, say with Import["!program", ...].
- The Wolfram Language supports asynchronous reading and writing of data. A simple case is URLSubmit, but ChannelListen, etc. allow you to set up a complete brokered publish-subscribe system.