For several reasons I was searching for a replacement for both Pocket and the Evernote web clipper. This resulted in a little iPad Shortcut that lets you download content from Safari on iPad and iPhone and turn iA Writer in a “read later app”. I’m very happy with it and you can download it here. I also wrote some words about how it works that you can read here.
7 Likes
Wow! I’ve done similarly just yesterday. I built a script in Pythonista specific for NYTimes Cooking and am saving as markdown through Shortcuts.
1 Like
This is awesome. Thank you. It would have never occurred to me to do this.
Is there any way to also include images so the images download as well?
1 Like
Why did you go for Phythonista? I haven’t used it but perhaps should give it a try. Is it hard to get started with it if you don’t have Python experience.
You’re welcome. It attaches images through links. iA Writer works with text and Markdown files that don’t include images like Pages or Word. There is an option to save images as assets that you can include in files, but I haven’t figured out yet how you can do that with Shortcuts.
Thats a good point. to be honest, just finished reading an article using your shortcut. It feels good to just read text without any distraction…
Thanks again for sharing this shortcut.
1 Like
It will show images when you enter reading mode (CMD + R). I just added that in the article too.
I needed to be able to parse html. Pythonista includes a package BeautifulSoup which makes it pretty easy to grab elements from html.
Pythonista is just an editor for python with automation capabilities. So, while you can use it to learn python, it would be very difficult to use without knowing python. But, I encourage you to learn it! Perhaps it’s an answer to doing the same action as you’ve done here on your Mac.
I've included my script with a lot of comments. Hopefully it will be a good reference!
import clipboard
import requests
import appex
import re
import html
import unicodedata
from bs4 import BeautifulSoup
# Shameless clone of https://github.com/davesrose/NY_Times_Cooking_Scraper in
# python
def clean(txt):
# Found this online, NYTimes includes '\xa0' which is raw ascii ' '
# character,
# this gets rid of that
return unicodedata.normalize("NFKD", txt.get_text().strip())
# This gets just the text from html
# element,
# and removes spaces in front/behind
# types of variables are not explicit in python, but they are enforced,
# - lists: a list of strings
# - title: a string
def ul(lists, title):
# f-strings allow you to insert variables inside of strings
# when you write out = ... this creates a variable called out
# it can then be used anywhere inside this function (def<name>())
# python figures out this is a String, so it will only allow you to
# do things strings can do.
out = f"## {title}\n\n"
for item in lists:
out += f"- {item}\n"
return out
# types of variables are not explicit in python, but they are enforced,
# - lists: a list of strings
# - title: a string
def ol(lists, title):
out = f"## {title}\n\n"
# enumerate gives us every item in a list and it's index
for idx, item in enumerate(lists):
out += f"{idx}. {item}\n"
return out
def main():
# This is mostly copied from the URL Extension template from Pythonista
# appex is the package which contains all the logic for being a share
# sheet extension
if not appex.is_running_extension():
print('Running in Pythonista app, using test data...\n')
url = 'http://example.com'
else:
# This gets the URL this extension is called on
url = appex.get_url()
if url:
# Pythonista cannot (or does not offer) getting the contents of a
# webpage of a URL.
# So we use the requests package to get the url this extension is
# called on
f = requests.get(url).text
# This tells BeautifulSoup to parse the html
content = BeautifulSoup(f, "html.parser")
# This means find the first element which has the class "recipe-title"
title = clean(content.find(class_="recipe-title"))
author = clean(content.find(class_="byline-name"))
yield_ = content.find(class_="recipe-time-yield")
# the class "recipe-time-yield" contains 2 li elements This is how you
# can manually move up/down/sideways in the parsed document
yield_servings = clean(yield_.li.span.next_sibling.next_sibling)
yield_time = clean(yield_.li.next_sibling.next_sibling.span.
next_sibling.next_sibling)
notes = []
# find_all get's all elements with this class, whereas find (above)
# just gets the first
for note in content.find_all(class_="recipe-note-description"):
notes.append(clean(note))
summary = clean(content.find(class_="topnote"))
units = []
for unit in content.find_all("span", class_="quantity"):
units.append(clean(unit))
names = []
for name in content.find_all("span", class_="ingredient-name"):
names.append(clean(name))
ingredients = []
# this loop goes from 0 to the number of units instead over
# each element in units
for i in range(0, len(units)):
# that allows us to use the same index in to the list for
# units AND names
ingredients.append(f"{units[i]} {names[i]}")
steps = []
step_container = content.find("ol", class_="recipe-steps")
# You can use find or find_all on the results of another find/find_all
for step in step_container.find_all("li"):
steps.append(clean(step))
image = content.find("meta", {"property": 'og:image'})['content']
# Here we use f-strings again to make beautiful markdown
as_markdown = f"""# {title}
By: {author}
{summary}
Yields: {yield_servings} in {yield_time}
{ul(notes,"Notes")}
{ul(ingredients, "Ingredients")}
{ol(steps, "Instructions")}
"""
# Pythonista cannot write outside of it's own folder
clipboard.set(as_markdown)
# this tells Pythonista this share sheet extension is done
appex.finish()
else:
print('No input URL found.')
# Don't worry about this :)
# But if you're curious,
# Everything we've written here is a function (def <name>())
# When python is run, it has no idea which one to run!
# This magic incantation tells python "This is where you start"
if __name__ == '__main__':
main()
# You can write anything you want down here
1 Like
This is fantastic. Solves a problem that’s been nagging me for a while. Works equally well with Pretext, which I use as my markdown/text app. Thanks a lot.
Nice, it will probably work with most apps that use text or markdown files.
Thanks, perhaps it’s time for me to play with Python. Starting with building some easy things that you can use helps to get going.
1 Like