blacklight/scribe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
scribe/spec/components/page_content_spec.cr

443 lines
11 KiB
Crystal
Raw Normal View History

First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
require "../spec_helper"
include Nodes
describe PageContent do
it "renders a single parent/child node structure" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Paragraph.new(children: [
Text.new(content: "hi"),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<p>hi</p>)
end
it "renders multiple childrens" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Paragraph.new(children: [
Text.new(content: "Hello, "),
Emphasis.new(children: [
Text.new(content: "World!"),
] of Child),
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
] of Child),
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
UnorderedList.new(children: [
ListItem.new(children: [
Text.new(content: "List!"),
] of Child),
ListItem.new(children: [
Text.new(content: "Again!"),
] of Child),
crystal tool format
2021-08-08 20:23:38 +02:00
] of Child),
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<p>Hello, <em>World!</em></p><ul><li>List!</li><li>Again!</li></ul>)
end
Render a Link Anchor As opposed to a user anchor
2021-07-04 23:06:17 +02:00
it "renders an anchor" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Anchor.new(children: [Text.new("link")] of Child, href: "https://example.com"),
] of Child
)
Render a Link Anchor As opposed to a user anchor
2021-07-04 23:06:17 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<a href="https://example.com">link</a>)
end
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
it "renders a blockquote" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
BlockQuote.new(children: [
Text.new("Wayne Gretzky. Michael Scott."),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
Fix Blockquotes In tufte.css blockquotes should contain a <p> that holds the content and an optional <footer> for the source of the quote. Otherwise the block quote text is unbounded and is way too wide. This wraps the content in a paragraph
2021-09-15 21:25:34 +02:00
html.should eq %(<blockquote><p>Wayne Gretzky. Michael Scott.</p></blockquote>)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
end
it "renders code" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Code.new(children: [
Text.new("foo = bar"),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<code>foo = bar</code>)
end
it "renders empasis" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Paragraph.new(children: [
Text.new(content: "This is "),
Emphasis.new(children: [
Text.new(content: "neat!"),
] of Child),
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
] of Child),
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<p>This is <em>neat!</em></p>)
end
Render image inside a figure with a caption Most images have a caption (or at least have the option of being captioned). Instead of displaying the raw image, it's not rendered inside a <figure> tag with a <figcaption> (possibly blank) as a sibling. The <figcaption> can be marked up with links.
2021-07-05 20:10:40 +02:00
it "renders a figure and figure caption" do
Display figure captions as margin notes On a thin viewport like a phone these show up as hidden at first until the user expands them by interacting with the "writing hand" icon. Each margin note needs a small bit of markup near it to enable the toggle. Each also needs a unique ID to ensure it doesn't interact with alternate content. The `hash` value of the FigureCaption's `children` provides this unique value.
2021-08-15 21:08:03 +02:00
children = [Text.new("A caption")] of Child
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Figure.new(children: [
Image.new(src: "image.png", originalWidth: 100, originalHeight: 200),
Display figure captions as margin notes On a thin viewport like a phone these show up as hidden at first until the user expands them by interacting with the "writing hand" icon. Each margin note needs a small bit of markup near it to enable the toggle. Each also needs a unique ID to ensure it doesn't interact with alternate content. The `hash` value of the FigureCaption's `children` provides this unique value.
2021-08-15 21:08:03 +02:00
FigureCaption.new(children: children),
Render image inside a figure with a caption Most images have a caption (or at least have the option of being captioned). Instead of displaying the raw image, it's not rendered inside a <figure> tag with a <figcaption> (possibly blank) as a sibling. The <figcaption> can be marked up with links.
2021-07-05 20:10:40 +02:00
] of Child),
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
] of Child
)
Render image inside a figure with a caption Most images have a caption (or at least have the option of being captioned). Instead of displaying the raw image, it's not rendered inside a <figure> tag with a <figcaption> (possibly blank) as a sibling. The <figcaption> can be marked up with links.
2021-07-05 20:10:40 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq stripped_html <<-HTML
<figure>
Don't render image heights explicitly The CSS itself will take care of scaling the image height based on the width. We still need to know the height to fetch the image because the height is in the URL, but we don't need to render it in the HTML.
2021-08-14 22:07:31 +02:00
<img src="https://cdn-images-1.medium.com/fit/c/100/200/image.png" width="100">
Display figure captions as margin notes On a thin viewport like a phone these show up as hidden at first until the user expands them by interacting with the "writing hand" icon. Each margin note needs a small bit of markup near it to enable the toggle. Each also needs a unique ID to ensure it doesn't interact with alternate content. The `hash` value of the FigureCaption's `children` provides this unique value.
2021-08-15 21:08:03 +02:00
<label class="margin-toggle" for="#{children.hash}">&#9997;&#xFE0E;</label>
<input class="margin-toggle" type="checkbox" id="#{children.hash}">
<span class="marginnote">
A caption
</span>
Render image inside a figure with a caption Most images have a caption (or at least have the option of being captioned). Instead of displaying the raw image, it's not rendered inside a <figure> tag with a <figcaption> (possibly blank) as a sibling. The <figcaption> can be marked up with links.
2021-07-05 20:10:40 +02:00
</figure>
HTML
end
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
it "renders a GitHub Gist" do
Proxy GitHub gists with rate limiting Previously, GitHub gists were embedded. The gist url would be detected in a paragraph and the page would render a script like: ```html <script src="https://gist.github.com/user/gist_id.js"></script> ``` The script would then embed the gist on the page. However, gists contain multiple files. It's technically possible to embed a single file in the same way by appending a `file` query param: ```html <script src="https://gist.github.com/user/gist_id.js?file=foo.txt"></script> ``` I wanted to try and tackle proxying gists instead. Overview -------- At a high level the PageConverter kicks off the work of fetching and storing the gist content, then sends that content down to the `ParagraphConverter`. When a paragraph comes up that contains a gist embed, it retrieves the previously fetched content. This allows all the necessary content to be fetched up front so the minimum number of requests need to be made. Fetching Gists -------------- There is now a `GithubClient` class that gets gist content from GitHub's ReST API. The gist API response looks something like this (non-relevant keys removed): ```json { "files": { "file-one.txt": { "filename": "file-one.txt", "raw_url": "https://gist.githubusercontent.com/<username>/<id>/raw/<file_id>/file-o ne.txt", "content": "..." }, "file-two.txt": { "filename": "file-two.txt", "raw_url": "https://gist.githubusercontent.com/<username>/<id>/raw/<file_id>/file-t wo.txt", "content": "..." } } } ``` That response gets turned into a bunch of `GistFile` objects that are then stored in a request-level `GistStore`. Crystal's JSON parsing does not make it easy to parse json with arbitrary keys into objects. This is because each key corresponds to an object property, like `property name : String`. If Crystal doesn't know the keys ahead of time, there's no way to know what methods to create. That's a problem here because the key for each gist file is the unique filename. Fortunately, the keys for each _file_ follows the same pattern and are easy to parse into a `GistFile` object. To turn gist file JSON into Crystal objects, the `GithubClient` turns the whole response into a `JSON::Any` which is like a Hash. Then it extracts just the file data objects and parses those into `GistFile` objects. Those `GistFile` objects are then cached in a `GistStore` that is shared for the page, which means one gist cache per request/article. `GistFile` objects can be fetched out of the store by file, or if no file is specified, it returns all files in the gist. The GistFile is rendered as a link of the file's name to the file in the gist on GitHub, and then a code block of the contents of the file. In summary, the `PageConverter`: * Scans the paragraphs for GitHub gists using `GistScanner` * Requests their data from GitHub using the `GithubClient` * Parses the response into `GistFile`s and populates the `GistStore` * Passes that `GistStore` to the `ParagraphConverter` to use when constructing the page nodes Caching ------- GitHub limits API requests to 5000/hour with a valid api token and 60/hour without. 60 is pretty tight for the usage that scribe.rip gets, but 5000 is reasonable most of the time. Not every article has an embedded gist, but some articles have multiple gists. A viral article (of which Scribe has seen two at the time of this commit) might receive a little over 127k hits/day, which is an average of over 5300/hour. If that article had a gist, Scribe would reach the API limit during parts of the day with high traffic. If it had multiple gists, it would hit it even more. However, average traffic is around 30k visits/day which would be well under the limit, assuming average load. To help not hit that limit, a `GistStore` holds all the `GistFile` objects per gist. The logic in `GistScanner` is smart enough to only return unique gist URLs so each gist is only requested once even if multiple files from one gist exist in an article. This limits the number of times Scribe hits the GitHub API. If Scribe is rate-limited, instead of populating a `GistCache` the `PageConverter` will create a `RateLimitedGistStore`. This is an object that acts like the `GistStore` but returns `RateLimitedGistFile` objects instead of `GistFile` objects. This allows Scribe to gracefully degrade in the event of reaching the rate limit. If rate-limiting becomes a regular problem, Scribe could also be reworked to fallback to the embedded gists again. API Credentials --------------- API credentials are in the form of a GitHub username and a personal access token attached to that username. To get a token, visit https://github.com/settings/tokens and create a new token. The only permission it needs is `gist`. This token is set via the `GITHUB_PERSONAL_ACCESS_TOKEN` environment variable. The username also needs to be set via `GITHUB_USERNAME`. When developing locally, these can both be set in the .env file. Authentication is probably not necessary locally, but it's there if you want to test. If either token is missing, unauthenticated requests are made. Rendering --------- The node tree itself holds a `GithubGist` object. It has a reference to the `GistStore` and the original gist URL. When it renders the page requests the gist's `files`. The gist ID and optional file are detected, and then used to request the file(s) from the `GistStore`. Gists render as a list of each files contents and a link to the file on GitHub. If the requests were rate limited, the store is a `RateLimitedGistStore` and the files are `RateLimitedGistFile`s. These rate-limited objects rendered with a link to the gist on GitHub and text saying that Scribe has been rate-limited. If somehow the file requested doesn't exist in the store, it displays similarly to the rate-limited file but with "file missing" text instead of "rate limited" text. GitHub API docs: https://docs.github.com/en/rest/reference/gists Rate Limiting docs: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate- limiting
2022-01-23 21:05:46 +01:00
store = GistStore.new
gist_file = GistFile.new(
filename: "example",
content: "content",
raw_url: "https://gist.githubusercontent.com/user/1/raw/abc/example"
)
store.store["1"] = [gist_file]
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
created_at: Time.local,
nodes: [
Proxy GitHub gists with rate limiting Previously, GitHub gists were embedded. The gist url would be detected in a paragraph and the page would render a script like: ```html <script src="https://gist.github.com/user/gist_id.js"></script> ``` The script would then embed the gist on the page. However, gists contain multiple files. It's technically possible to embed a single file in the same way by appending a `file` query param: ```html <script src="https://gist.github.com/user/gist_id.js?file=foo.txt"></script> ``` I wanted to try and tackle proxying gists instead. Overview -------- At a high level the PageConverter kicks off the work of fetching and storing the gist content, then sends that content down to the `ParagraphConverter`. When a paragraph comes up that contains a gist embed, it retrieves the previously fetched content. This allows all the necessary content to be fetched up front so the minimum number of requests need to be made. Fetching Gists -------------- There is now a `GithubClient` class that gets gist content from GitHub's ReST API. The gist API response looks something like this (non-relevant keys removed): ```json { "files": { "file-one.txt": { "filename": "file-one.txt", "raw_url": "https://gist.githubusercontent.com/<username>/<id>/raw/<file_id>/file-o ne.txt", "content": "..." }, "file-two.txt": { "filename": "file-two.txt", "raw_url": "https://gist.githubusercontent.com/<username>/<id>/raw/<file_id>/file-t wo.txt", "content": "..." } } } ``` That response gets turned into a bunch of `GistFile` objects that are then stored in a request-level `GistStore`. Crystal's JSON parsing does not make it easy to parse json with arbitrary keys into objects. This is because each key corresponds to an object property, like `property name : String`. If Crystal doesn't know the keys ahead of time, there's no way to know what methods to create. That's a problem here because the key for each gist file is the unique filename. Fortunately, the keys for each _file_ follows the same pattern and are easy to parse into a `GistFile` object. To turn gist file JSON into Crystal objects, the `GithubClient` turns the whole response into a `JSON::Any` which is like a Hash. Then it extracts just the file data objects and parses those into `GistFile` objects. Those `GistFile` objects are then cached in a `GistStore` that is shared for the page, which means one gist cache per request/article. `GistFile` objects can be fetched out of the store by file, or if no file is specified, it returns all files in the gist. The GistFile is rendered as a link of the file's name to the file in the gist on GitHub, and then a code block of the contents of the file. In summary, the `PageConverter`: * Scans the paragraphs for GitHub gists using `GistScanner` * Requests their data from GitHub using the `GithubClient` * Parses the response into `GistFile`s and populates the `GistStore` * Passes that `GistStore` to the `ParagraphConverter` to use when constructing the page nodes Caching ------- GitHub limits API requests to 5000/hour with a valid api token and 60/hour without. 60 is pretty tight for the usage that scribe.rip gets, but 5000 is reasonable most of the time. Not every article has an embedded gist, but some articles have multiple gists. A viral article (of which Scribe has seen two at the time of this commit) might receive a little over 127k hits/day, which is an average of over 5300/hour. If that article had a gist, Scribe would reach the API limit during parts of the day with high traffic. If it had multiple gists, it would hit it even more. However, average traffic is around 30k visits/day which would be well under the limit, assuming average load. To help not hit that limit, a `GistStore` holds all the `GistFile` objects per gist. The logic in `GistScanner` is smart enough to only return unique gist URLs so each gist is only requested once even if multiple files from one gist exist in an article. This limits the number of times Scribe hits the GitHub API. If Scribe is rate-limited, instead of populating a `GistCache` the `PageConverter` will create a `RateLimitedGistStore`. This is an object that acts like the `GistStore` but returns `RateLimitedGistFile` objects instead of `GistFile` objects. This allows Scribe to gracefully degrade in the event of reaching the rate limit. If rate-limiting becomes a regular problem, Scribe could also be reworked to fallback to the embedded gists again. API Credentials --------------- API credentials are in the form of a GitHub username and a personal access token attached to that username. To get a token, visit https://github.com/settings/tokens and create a new token. The only permission it needs is `gist`. This token is set via the `GITHUB_PERSONAL_ACCESS_TOKEN` environment variable. The username also needs to be set via `GITHUB_USERNAME`. When developing locally, these can both be set in the .env file. Authentication is probably not necessary locally, but it's there if you want to test. If either token is missing, unauthenticated requests are made. Rendering --------- The node tree itself holds a `GithubGist` object. It has a reference to the `GistStore` and the original gist URL. When it renders the page requests the gist's `files`. The gist ID and optional file are detected, and then used to request the file(s) from the `GistStore`. Gists render as a list of each files contents and a link to the file on GitHub. If the requests were rate limited, the store is a `RateLimitedGistStore` and the files are `RateLimitedGistFile`s. These rate-limited objects rendered with a link to the gist on GitHub and text saying that Scribe has been rate-limited. If somehow the file requested doesn't exist in the store, it displays similarly to the rate-limited file but with "file missing" text instead of "rate limited" text. GitHub API docs: https://docs.github.com/en/rest/reference/gists Rate Limiting docs: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate- limiting
2022-01-23 21:05:46 +01:00
GithubGist.new(href: "https://gist.github.com/user/1", gist_store: store),
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
] of Child
)
html = PageContent.new(page: page).render_to_string
html.should eq stripped_html <<-HTML
Proxy GitHub gists with rate limiting Previously, GitHub gists were embedded. The gist url would be detected in a paragraph and the page would render a script like: ```html <script src="https://gist.github.com/user/gist_id.js"></script> ``` The script would then embed the gist on the page. However, gists contain multiple files. It's technically possible to embed a single file in the same way by appending a `file` query param: ```html <script src="https://gist.github.com/user/gist_id.js?file=foo.txt"></script> ``` I wanted to try and tackle proxying gists instead. Overview -------- At a high level the PageConverter kicks off the work of fetching and storing the gist content, then sends that content down to the `ParagraphConverter`. When a paragraph comes up that contains a gist embed, it retrieves the previously fetched content. This allows all the necessary content to be fetched up front so the minimum number of requests need to be made. Fetching Gists -------------- There is now a `GithubClient` class that gets gist content from GitHub's ReST API. The gist API response looks something like this (non-relevant keys removed): ```json { "files": { "file-one.txt": { "filename": "file-one.txt", "raw_url": "https://gist.githubusercontent.com/<username>/<id>/raw/<file_id>/file-o ne.txt", "content": "..." }, "file-two.txt": { "filename": "file-two.txt", "raw_url": "https://gist.githubusercontent.com/<username>/<id>/raw/<file_id>/file-t wo.txt", "content": "..." } } } ``` That response gets turned into a bunch of `GistFile` objects that are then stored in a request-level `GistStore`. Crystal's JSON parsing does not make it easy to parse json with arbitrary keys into objects. This is because each key corresponds to an object property, like `property name : String`. If Crystal doesn't know the keys ahead of time, there's no way to know what methods to create. That's a problem here because the key for each gist file is the unique filename. Fortunately, the keys for each _file_ follows the same pattern and are easy to parse into a `GistFile` object. To turn gist file JSON into Crystal objects, the `GithubClient` turns the whole response into a `JSON::Any` which is like a Hash. Then it extracts just the file data objects and parses those into `GistFile` objects. Those `GistFile` objects are then cached in a `GistStore` that is shared for the page, which means one gist cache per request/article. `GistFile` objects can be fetched out of the store by file, or if no file is specified, it returns all files in the gist. The GistFile is rendered as a link of the file's name to the file in the gist on GitHub, and then a code block of the contents of the file. In summary, the `PageConverter`: * Scans the paragraphs for GitHub gists using `GistScanner` * Requests their data from GitHub using the `GithubClient` * Parses the response into `GistFile`s and populates the `GistStore` * Passes that `GistStore` to the `ParagraphConverter` to use when constructing the page nodes Caching ------- GitHub limits API requests to 5000/hour with a valid api token and 60/hour without. 60 is pretty tight for the usage that scribe.rip gets, but 5000 is reasonable most of the time. Not every article has an embedded gist, but some articles have multiple gists. A viral article (of which Scribe has seen two at the time of this commit) might receive a little over 127k hits/day, which is an average of over 5300/hour. If that article had a gist, Scribe would reach the API limit during parts of the day with high traffic. If it had multiple gists, it would hit it even more. However, average traffic is around 30k visits/day which would be well under the limit, assuming average load. To help not hit that limit, a `GistStore` holds all the `GistFile` objects per gist. The logic in `GistScanner` is smart enough to only return unique gist URLs so each gist is only requested once even if multiple files from one gist exist in an article. This limits the number of times Scribe hits the GitHub API. If Scribe is rate-limited, instead of populating a `GistCache` the `PageConverter` will create a `RateLimitedGistStore`. This is an object that acts like the `GistStore` but returns `RateLimitedGistFile` objects instead of `GistFile` objects. This allows Scribe to gracefully degrade in the event of reaching the rate limit. If rate-limiting becomes a regular problem, Scribe could also be reworked to fallback to the embedded gists again. API Credentials --------------- API credentials are in the form of a GitHub username and a personal access token attached to that username. To get a token, visit https://github.com/settings/tokens and create a new token. The only permission it needs is `gist`. This token is set via the `GITHUB_PERSONAL_ACCESS_TOKEN` environment variable. The username also needs to be set via `GITHUB_USERNAME`. When developing locally, these can both be set in the .env file. Authentication is probably not necessary locally, but it's there if you want to test. If either token is missing, unauthenticated requests are made. Rendering --------- The node tree itself holds a `GithubGist` object. It has a reference to the `GistStore` and the original gist URL. When it renders the page requests the gist's `files`. The gist ID and optional file are detected, and then used to request the file(s) from the `GistStore`. Gists render as a list of each files contents and a link to the file on GitHub. If the requests were rate limited, the store is a `RateLimitedGistStore` and the files are `RateLimitedGistFile`s. These rate-limited objects rendered with a link to the gist on GitHub and text saying that Scribe has been rate-limited. If somehow the file requested doesn't exist in the store, it displays similarly to the rate-limited file but with "file missing" text instead of "rate limited" text. GitHub API docs: https://docs.github.com/en/rest/reference/gists Rate Limiting docs: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate- limiting
2022-01-23 21:05:46 +01:00
<p>
<code>
<a href="https://gist.github.com/user/1#file-example">example</a>
</code>
</p>
<pre class="gist">
<code>content</code>
</pre>
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
HTML
end
Parsing Fix: Add H2 Paragraph type The post id 34dead42a28 contained a new paragraph type: H2. Previously the only known header types were H3 and H4. In this case, the paragraph doesn't actually get rendered because it's the page title which is removed from the page nodes (see commits 6baba803 and then fba87c10). However, it somehow an author is able to get an H2 paragraph into the page, it will display as an <h1> just as H3 displays as <h2> and H4 displays as <h3>.
2021-10-16 22:19:44 +02:00
it "renders an H2" do
page = Page.new(
title: "Title",
author: user_anchor_factory,
created_at: Time.local,
nodes: [
Heading1.new(children: [
Text.new(content: "Title!"),
Add unique ID to headings The `name` field on the `paragraph` type contains a unique ID for the paragraph. It's not guaranteed to be there, on images for example like in the `fd8d091ab8ef` post, but it's there for everything else I can find. This enables deep linking. There's no way to get to the deep link other than opening up the web console. I wanted to link every heading, but you can actually have links in part of a heading so that's not tenable. Maybe a "permalink" link next to every heading?
2023-03-25 16:16:00 +01:00
] of Child, identifier: "ab12"),
Parsing Fix: Add H2 Paragraph type The post id 34dead42a28 contained a new paragraph type: H2. Previously the only known header types were H3 and H4. In this case, the paragraph doesn't actually get rendered because it's the page title which is removed from the page nodes (see commits 6baba803 and then fba87c10). However, it somehow an author is able to get an H2 paragraph into the page, it will display as an <h1> just as H3 displays as <h2> and H4 displays as <h3>.
2021-10-16 22:19:44 +02:00
] of Child
)
html = PageContent.new(page: page).render_to_string
Add unique ID to headings The `name` field on the `paragraph` type contains a unique ID for the paragraph. It's not guaranteed to be there, on images for example like in the `fd8d091ab8ef` post, but it's there for everything else I can find. This enables deep linking. There's no way to get to the deep link other than opening up the web console. I wanted to link every heading, but you can actually have links in part of a heading so that's not tenable. Maybe a "permalink" link next to every heading?
2023-03-25 16:16:00 +01:00
html.should eq %(<h1 id="ab12">Title!</h1>)
Parsing Fix: Add H2 Paragraph type The post id 34dead42a28 contained a new paragraph type: H2. Previously the only known header types were H3 and H4. In this case, the paragraph doesn't actually get rendered because it's the page title which is removed from the page nodes (see commits 6baba803 and then fba87c10). However, it somehow an author is able to get an H2 paragraph into the page, it will display as an <h1> just as H3 displays as <h2> and H4 displays as <h3>.
2021-10-16 22:19:44 +02:00
end
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
it "renders an H3" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Heading2.new(children: [
Text.new(content: "Title!"),
Add unique ID to headings The `name` field on the `paragraph` type contains a unique ID for the paragraph. It's not guaranteed to be there, on images for example like in the `fd8d091ab8ef` post, but it's there for everything else I can find. This enables deep linking. There's no way to get to the deep link other than opening up the web console. I wanted to link every heading, but you can actually have links in part of a heading so that's not tenable. Maybe a "permalink" link next to every heading?
2023-03-25 16:16:00 +01:00
] of Child, identifier: "ab12"),
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
Add unique ID to headings The `name` field on the `paragraph` type contains a unique ID for the paragraph. It's not guaranteed to be there, on images for example like in the `fd8d091ab8ef` post, but it's there for everything else I can find. This enables deep linking. There's no way to get to the deep link other than opening up the web console. I wanted to link every heading, but you can actually have links in part of a heading so that's not tenable. Maybe a "permalink" link next to every heading?
2023-03-25 16:16:00 +01:00
html.should eq %(<h2 id="ab12">Title!</h2>)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
end
it "renders an H4" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Heading3.new(children: [
Text.new(content: "In Conclusion..."),
Add unique ID to headings The `name` field on the `paragraph` type contains a unique ID for the paragraph. It's not guaranteed to be there, on images for example like in the `fd8d091ab8ef` post, but it's there for everything else I can find. This enables deep linking. There's no way to get to the deep link other than opening up the web console. I wanted to link every heading, but you can actually have links in part of a heading so that's not tenable. Maybe a "permalink" link next to every heading?
2023-03-25 16:16:00 +01:00
] of Child, identifier: "ab12"),
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
Add unique ID to headings The `name` field on the `paragraph` type contains a unique ID for the paragraph. It's not guaranteed to be there, on images for example like in the `fd8d091ab8ef` post, but it's there for everything else I can find. This enables deep linking. There's no way to get to the deep link other than opening up the web console. I wanted to link every heading, but you can actually have links in part of a heading so that's not tenable. Maybe a "permalink" link next to every heading?
2023-03-25 16:16:00 +01:00
html.should eq %(<h3 id="ab12">In Conclusion...</h3>)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
end
it "renders an image" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Paragraph.new(children: [
Image.new(src: "image.png", originalWidth: 100, originalHeight: 200),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
Fetch the resized image Instead of getting the full size image, the image can be fetched with a width and height parameter so that only the resized data is transferred. The url looks like this: https://cdn-images-1.medium.com/fit/c/<width>/<height>/<media-id> I picked a max image width of 800px. If the image width is more than that, it scales the width down to 800, then applies that ratio to the height. If it's smaller than that, the image is displayed as the original.
2021-07-05 20:54:58 +02:00
html.should eq stripped_html <<-HTML
<p>
Don't render image heights explicitly The CSS itself will take care of scaling the image height based on the width. We still need to know the height to fetch the image because the height is in the URL, but we don't need to render it in the HTML.
2021-08-14 22:07:31 +02:00
<img src="https://cdn-images-1.medium.com/fit/c/100/200/image.png" width="100">
Fetch the resized image Instead of getting the full size image, the image can be fetched with a width and height parameter so that only the resized data is transferred. The url looks like this: https://cdn-images-1.medium.com/fit/c/<width>/<height>/<media-id> I picked a max image width of 800px. If the image width is more than that, it scales the width down to 800, then applies that ratio to the height. If it's smaller than that, the image is displayed as the original.
2021-07-05 20:54:58 +02:00
</p>
HTML
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
end
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
it "renders embedded content" do
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
created_at: Time.local,
nodes: [
EmbeddedContent.new(
src: "https://example.com",
originalWidth: 1000,
originalHeight: 600,
),
] of Child
)
html = PageContent.new(page: page).render_to_string
html.should eq stripped_html <<-HTML
Embed widths are now the same width as all content
2021-09-25 19:26:10 +02:00
<figure>
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
<iframe src="https://example.com" width="800" height="480" frameborder="0" allowfullscreen="true">
</iframe>
Embed widths are now the same width as all content
2021-09-25 19:26:10 +02:00
</figure>
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
HTML
end
it "renders an embedded link container" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Paragraph.new(children: [
Render embedded content PostResponse::Paragraph's that are of type IFRAME have extra data in the iframe attribute to specify what's in the iframe. Not all data is the same, however. I've identified three types and am using the new EmbeddedConverter class to convert them: * EmbeddedContent, the full iframe experience * GithubGist, because medium or github treat embeds differently for whatever reason * EmbeddedLink, the old style, just a link to the content. Effectively a fallback The size of the original iframe is also specified as an attribute. This code resizes it. The resizing is determined by figuring out the width/height ratio and setting the width to 800. EmbeddedContent can be displayed if we have an embed.ly url, which most iframe response data has. GitHub gists are a notable exception. Gists instead can be embedded simply by taking the gist URL and attaching .js to the end. That becomes the iframe's src attribute. The PostResponse::Paragraph's iframe attribute is nillable. Previous code used lots of if-statements with variable bindings to work with the possible nil values: ```crystal if foo = obj.nillable_value # obj.nillable_value was not nil and foo contains the value else # obj.nillable_value was nil so do something else end ``` See https://crystal-lang.org/reference/syntax_and_semantics/if_var.html for more info In the EmbeddedConverter the monads library has been introduced to get rid of at least one level of nillability. This wraps values in Maybe which allows for a cleaner interface: ```crystal Monads::Try(Value).new(->{ obj.nillable_value }) .to_maybe .fmap(->(value: Value) { # do something with value }) .value_or(# value was nil, do something else) ``` This worked to get the iframe attribute from a Paragraph: ```crystal Monads::Try(PostResponse::IFrame).new(->{ paragraph.iframe }) .to_maybe .fmap(->(iframe : PostResponse::IFrame) { # iframe is not nil! }) .fmap(#and so on) .value_or(Empty.new) ``` iframe only has one attribute: mediaResource which contains the iframe data. That was used to determine one of the three types above. Finally, Tufte.css has options for iframes. They mostly look good except for tweets which are too small and weirdly in the center of the page which actually looks off-center. That's for another day though.
2021-09-13 19:27:52 +02:00
EmbeddedLink.new(href: "https://example.com"),
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
Show the host for the iframe link Instead of showing only: Click to visit embedded content An embedded link now displays with the domain it's linking to: Embedded content at example.com This hopefully breaks up the links a bit so it'e easier to distinguish between a bunch of them in a row (as long as they are on different domains).
2021-07-05 21:36:38 +02:00
html.should eq stripped_html <<-HTML
<p>
Embed widths are now the same width as all content
2021-09-25 19:26:10 +02:00
<figure>
Show the host for the iframe link Instead of showing only: Click to visit embedded content An embedded link now displays with the domain it's linking to: Embedded content at example.com This hopefully breaks up the links a bit so it'e easier to distinguish between a bunch of them in a row (as long as they are on different domains).
2021-07-05 21:36:38 +02:00
<a href="https://example.com">Embedded content at example.com</a>
Embed widths are now the same width as all content
2021-09-25 19:26:10 +02:00
</figure>
Show the host for the iframe link Instead of showing only: Click to visit embedded content An embedded link now displays with the domain it's linking to: Embedded content at example.com This hopefully breaks up the links a bit so it'e easier to distinguish between a bunch of them in a row (as long as they are on different domains).
2021-07-05 21:36:38 +02:00
</p>
HTML
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
end
Add MIXTAPE_EMBED paragraph type
2021-09-08 03:13:28 +02:00
it "renders an mixtape embed container" do
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add MIXTAPE_EMBED paragraph type
2021-09-08 03:13:28 +02:00
created_at: Time.local,
nodes: [
Paragraph.new(children: [
MixtapeEmbed.new(children: [
Anchor.new(
children: [Text.new("Mixtape")] of Child,
href: "https://example.com"
),
] of Child),
] of Child),
] of Child
)
html = PageContent.new(page: page).render_to_string
html.should eq stripped_html <<-HTML
<p>
Embed widths are now the same width as all content
2021-09-25 19:26:10 +02:00
<blockquote>
<p>
<a href="https://example.com">Mixtape</a>
</p>
</blockquote>
Add MIXTAPE_EMBED paragraph type
2021-09-08 03:13:28 +02:00
</p>
HTML
end
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
it "renders an ordered list" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
OrderedList.new(children: [
ListItem.new(children: [Text.new("One")] of Child),
ListItem.new(children: [Text.new("Two")] of Child),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<ol><li>One</li><li>Two</li></ol>)
end
it "renders an preformatted text" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Paragraph.new(children: [
Text.new("Hello, world!"),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<p>Hello, world!</p>)
end
it "renders an preformatted text" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Preformatted.new(children: [
Text.new("New\nline"),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<pre>New\nline</pre>)
end
it "renders strong text" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Strong.new(children: [
Text.new("Oh yeah!"),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<strong>Oh yeah!</strong>)
end
it "renders an unordered list" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
UnorderedList.new(children: [
ListItem.new(children: [Text.new("Apple")] of Child),
ListItem.new(children: [Text.new("Banana")] of Child),
] of Child),
] of Child
)
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<ul><li>Apple</li><li>Banana</li></ul>)
end
Render a User Anchor
2021-07-04 23:37:45 +02:00
it "renders a user anchor" do
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
page = Page.new(
title: "Title",
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
author: user_anchor_factory,
Add post creation date/time
2021-09-04 23:32:27 +02:00
created_at: Time.local,
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
nodes: [
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
UserAnchor.new(children: [Text.new("Some User")] of Child, user_id: "abc123"),
Extract tile and subtitle from initial paragraphs Medium guides each post to have a Title and Subtitle. They are rendered as the first two paragraphs: H3 and H4 respectively. If they exist, a new PageConverter class extracts them and sets them on the page. However, they aren't required. If the first two paragraphs aren't H3 and H4, the PageConverter falls back to using the first paragraph as the title, and setting the subtitle to blank. The remaining paragraphs are passed into the ParagraphConverter as normal.
2021-08-14 23:36:10 +02:00
] of Child
)
Render a User Anchor
2021-07-04 23:37:45 +02:00
html = PageContent.new(page: page).render_to_string
html.should eq %(<a href="https://medium.com/u/abc123">Some User</a>)
end
First step rendering a page The API responds with a bunch of paragraphs which the client converts into Paragraph objects. This turns the paragraphs in a PostResponse's Paragraph objects into the form needed to render them on a page. This includes converting flat list elements into list elements nested by a UL. And adding a limited markups along the way. The array of paragraphs is passed to a recursive function. The function takes the first paragraph and either wraps the (marked up) contents in a container tag (like Paragraph or Heading3), and then moves onto the next tag. If it finds a list, it starts parsing the next paragraphs as a list instead. Originally, this was implemented like so: ```crystal paragraph = paragraphs.shift if list? convert_list([paragraph] + paragraphs) end ``` However, passing the `paragraphs` after adding it to the already shifted `paragraph` creates a new object. This means `paragraphs` won't be mutated and once the list is parsed, it starts with the next element of the list. Instead, the element is `shift`ed inside each converter. ```crystal if paragraphs.first == list? convert_list(paragraphs) end def convert_list(paragraphs) paragraph = paragraphs.shift # ... end ``` When rendering, there is an Empty and Container object. These represent a kind of "null object" for both leafs and parent objects respectively. They should never actually render. Emptys are filtered out, and Containers are never created explicitly but this will make the types pass. IFrames are a bit of a special case. Each IFrame has custom data on it that this system would need to be aware of. For now, instead of trying to parse the seemingly large number of iframe variations and dealing with embedded iframe problems, this will just keep track of the source page URL and send the user there with a link.
2021-05-16 20:14:25 +02:00
end
Render image inside a figure with a caption Most images have a caption (or at least have the option of being captioned). Instead of displaying the raw image, it's not rendered inside a <figure> tag with a <figcaption> (possibly blank) as a sibling. The <figcaption> can be marked up with links.
2021-07-05 20:10:40 +02:00
def stripped_html(html : String)
html.gsub(/\n\s*/, "").strip
end
Link to the author's page Right now this links to the user's medium page. It may link to an internal page in the future. Instead of the Page taking the author as a string, it now takes a PostResponse::Creator object. The Articles::ShowPage then converts the Creator (a name and user_id) to an author link. Finally, I did some refactoring of UserAnchor (which I thought I was going to use for this) to change it's userId attribute to user_id as is Crystal convention.
2021-09-15 21:44:28 +02:00
def user_anchor_factory(username = "someone", user_id = "abc123")
PostResponse::Creator.from_json <<-JSON
{
"id": "#{user_id}",
"name": "#{username}"
}
JSON
end
Reference in a new issue Copy permalink