An Interest In:
Web News this Week
- April 20, 2024
- April 19, 2024
- April 18, 2024
- April 17, 2024
- April 16, 2024
- April 15, 2024
- April 14, 2024
Attempting to Learn Go - Ghost to Hugo 2
Intro
Welcome back! We are continuing on our journey to make a prototype program that converts an exported Ghost database to Markdown. With the end goal being that we can get shindakun.net up and running with Hugo. Last time, we took it pretty easy and focused mostly on reading the file into memory and converting the JSON to a Go struct. From there we printed out the first post.
Post Data
As a recap here is what one of the post fields contain.
{ "id": "60710b90705967038fe662d6", "uuid": "71ba3d71-ac18-4f33-82f7-1962baa83a07", "title": "db test", "slug": "db-test", "mobiledoc": "{\"version\":\"0.3.1\",\"markups\":[],\"atoms\":[],\"cards\":[[\"markdown\",{\"cardName\":\"card-markdown\",\"markdown\":\"\\n<strike>This is a db test</strike>.\\n\"}]],\"sections\":[[10,0]],\"ghostVersion\":\"3.0\"}", "html": "<!--kg-card-begin: markdown--><p><strike>This is a db test</strike>.</p>
<!--kg-card-end: markdown-->", "comment_id": "2", "plaintext": "This is a db test.", "feature_image": null, "featured": 0, "type": "post", "status": "published", "locale": null, "visibility": "public", "email_recipient_filter": "none", "author_id": "60710b8d705967038fe66214", "created_at": "2004-08-09T19:11:20.000Z", "updated_at": "2004-08-09T19:11:20.000Z", "published_at": "2004-08-09T19:11:20.000Z", "custom_excerpt": null, "codeinjection_head": null, "codeinjection_foot": null, "custom_template": null, "canonical_url": null},
The majority of the fields we need to craft a post with appropriate frontmatter exist within this the object were getting back. We can see the title, slug, published date, etc. The section that contains the Markdown is in a format known as Mobiledoc.
Mobiledoc
According to the Ghost documentation Mobiledoc is
a standardised JSON-based document storage format, which forms the heart of publishing with Ghost.
When extracted from the JSON object and cleaned up well have another bit of JSON we can work with.
{ "version": "0.3.1", "markups": [], "atoms": [], "cards": [ [ "markdown", { "cardName": "card-markdown", "markdown": "
<strike>This is a db test</strike>.
" } ] ], "sections": [ [ 10, 0 ] ], "ghostVersion": "3.0"}
Sounds Easy
First well use our favorite site, the JSON to Go converter to convert the JSON object to a struct we can work with.
type Mobiledoc struct { Version string `json:"version"` Markups []interface{} `json:"markups"` Atoms []interface{} `json:"atoms"` Cards [][]interface{} `json:"cards"` Sections [][]int `json:"sections"` GhostVersion string `json:"ghostVersion"`}
Our code is pretty long now so Im going to leave out the other struct GhostDatabase
struct, itll be in the complete code listing below though. Were still going to be dumping code to the screen since were still working on our decoding logic.
package mainimport ( "encoding/json" "fmt" "io" "os" "strconv" "time")type GhostDatabase struct {...}type Mobiledoc struct { Version string `json:"version"` Markups []interface{} `json:"markups"` Atoms []interface{} `json:"atoms"` Cards [][]interface{} `json:"cards"` Sections [][]int `json:"sections"` GhostVersion string `json:"ghostVersion"`}func main() { fmt.Println("ghost2hugo") file, err := os.Open("shindakun-dot-net.ghost.2022-03-18-22-02-58.json") if err != nil { fmt.Println(err) } defer file.Close() b, err := io.ReadAll(file) if err != nil { fmt.Println(err) } var db GhostDatabase err = json.Unmarshal(b, &db) if err != nil { fmt.Println(err) }
Lets continue to focus on the first post for now since once we have that working it should just be a matter of looping through the database. This is where it gets a little tricky. Were working with a couple of nested arrays so to work our way down to the appropriate section we use db.Db[0].Data.Posts[0].Mobiledoc
. This will give use the escaped version of our JSON object.
"{\"version\":\"0.3.1\",\"markups\":[],\"atoms\":[],\"cards\":[[\"markdown\",{\"cardName\":\"card-markdown\",\"markdown\":\"\\n<strike>This is a db test</strike>.\\n\"}]],\"sections\":[[10,0]],\"ghostVersion\":\"3.0\"}"
Lets Go On A Trip
I knew that there would be a way to unescape the string that we get, checking the Go documentation led me to strconv.Unquote
.
Unquote interprets s as a single-quoted, double-quoted, or backquoted Go string literal, returning the string value that s quotes. (If s is single-quoted, it would be a Go character literal; Unquote returns the corresponding one-character string.)
Exactly what we need! Except that when I tried to unquote the string I kept receiving an invalid syntax
error. This had me confused for a little bit. After puzzling over it I realized if I prepended and appended a back tick to the string it seems to be treated as a raw string literal. This leads to code that looks like the following.
c := "`" + db.Db[0].Data.Posts[0].Mobiledoc + "`" un, err := strconv.Unquote(c) if err != nil { fmt.Println(err) } fmt.Printf("%v
", un)
Finally! We have the JSON!
{"version":"0.3.1","markups":[],"atoms":[],"cards":[["markdown",{"cardName":"card-markdown","markdown":"
<strike>This is a db test</strike>.
"}]],"sections":[[10,0]],"ghostVersion":"3.0"}
Now, we can unmarshal that into the Mobiledoc struct we set up earlier!
var md Mobiledoc err = json.Unmarshal([]byte(un), &md) if err != nil { fmt.Println(err) } fmt.Printf("%#v", md)
Checking our dumped result we can see that weve got the expected data.
main.Mobiledoc{Version:"0.3.1", Markups:[]interface {}{}, Atoms:[]interface {}{}, Cards:[][]interface {}{[]interface {}{"markdown", map[string]interface {}{"cardName":"card-markdown", "markdown":"
<strike>This is a db test</strike>.
"}}}, Sections:[][]int{[]int{10, 0}}, GhostVersion:"3.0"}
Right now were concerned with the section called Cards
which if we naively want to access we can use the following.
card := md.Cards[0][1] fmt.Printf("
card: %#v
", card)
card: map[string]interface {}{"cardName":"card-markdown", "markdown":"
<strike>This is a db test</strike>.
"}
Lets convert that to something a bit easier to access.
bbb := card.(map[string]interface{}) fmt.Println(bbb["markdown"])
<strike>This is a db test</strike>.
Next Time
So far so good, weve extracted the Markdown from the first post as expected. Next time around well be writing a small loop to start trying to extract all the posts. I have a feeling thats where the fun will begin.
Enjoy this post? |
---|
How about buying me a coffee? |
Code Listing
package mainimport ( "encoding/json" "fmt" "io" "os" "strconv" "time")type GhostDatabase struct { Db []struct { Meta struct { ExportedOn int64 `json:"exported_on"` Version string `json:"version"` } `json:"meta"` Data struct { Posts []struct { ID string `json:"id"` UUID string `json:"uuid"` Title string `json:"title"` Slug string `json:"slug"` Mobiledoc string `json:"mobiledoc"` HTML string `json:"html"` CommentID string `json:"comment_id"` Plaintext string `json:"plaintext"` FeatureImage interface{} `json:"feature_image"` Featured int `json:"featured"` Type string `json:"type"` Status string `json:"status"` Locale interface{} `json:"locale"` Visibility string `json:"visibility"` EmailRecipientFilter string `json:"email_recipient_filter"` AuthorID string `json:"author_id"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` PublishedAt time.Time `json:"published_at"` CustomExcerpt interface{} `json:"custom_excerpt"` CodeinjectionHead interface{} `json:"codeinjection_head"` CodeinjectionFoot interface{} `json:"codeinjection_foot"` CustomTemplate interface{} `json:"custom_template"` CanonicalURL interface{} `json:"canonical_url"` } `json:"posts"` PostsAuthors []struct { ID string `json:"id"` PostID string `json:"post_id"` AuthorID string `json:"author_id"` SortOrder int `json:"sort_order"` } `json:"posts_authors"` PostsMeta []interface{} `json:"posts_meta"` PostsTags []struct { ID string `json:"id"` PostID string `json:"post_id"` TagID string `json:"tag_id"` SortOrder int `json:"sort_order"` } `json:"posts_tags"` Roles []struct { ID string `json:"id"` Name string `json:"name"` Description string `json:"description"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } `json:"roles"` RolesUsers []struct { ID string `json:"id"` RoleID string `json:"role_id"` UserID string `json:"user_id"` } `json:"roles_users"` Settings []struct { ID string `json:"id"` Group string `json:"group"` Key string `json:"key"` Value string `json:"value"` Type string `json:"type"` Flags interface{} `json:"flags"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } `json:"settings"` Tags []struct { ID string `json:"id"` Name string `json:"name"` Slug string `json:"slug"` Description interface{} `json:"description"` FeatureImage interface{} `json:"feature_image"` ParentID interface{} `json:"parent_id"` Visibility string `json:"visibility"` OgImage interface{} `json:"og_image"` OgTitle interface{} `json:"og_title"` OgDescription interface{} `json:"og_description"` TwitterImage interface{} `json:"twitter_image"` TwitterTitle interface{} `json:"twitter_title"` TwitterDescription interface{} `json:"twitter_description"` MetaTitle interface{} `json:"meta_title"` MetaDescription interface{} `json:"meta_description"` CodeinjectionHead interface{} `json:"codeinjection_head"` CodeinjectionFoot interface{} `json:"codeinjection_foot"` CanonicalURL interface{} `json:"canonical_url"` AccentColor interface{} `json:"accent_color"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } `json:"tags"` Users []struct { ID string `json:"id"` Name string `json:"name"` Slug string `json:"slug"` Password string `json:"password"` Email string `json:"email"` ProfileImage string `json:"profile_image"` CoverImage interface{} `json:"cover_image"` Bio interface{} `json:"bio"` Website interface{} `json:"website"` Location interface{} `json:"location"` Facebook interface{} `json:"facebook"` Twitter interface{} `json:"twitter"` Accessibility string `json:"accessibility"` Status string `json:"status"` Locale interface{} `json:"locale"` Visibility string `json:"visibility"` MetaTitle interface{} `json:"meta_title"` MetaDescription interface{} `json:"meta_description"` Tour interface{} `json:"tour"` LastSeen time.Time `json:"last_seen"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } `json:"users"` } `json:"data"` } `json:"db"`}type Mobiledoc struct { Version string `json:"version"` Markups []interface{} `json:"markups"` Atoms []interface{} `json:"atoms"` Cards [][]interface{} `json:"cards"` Sections [][]int `json:"sections"` GhostVersion string `json:"ghostVersion"`}func main() { fmt.Println("ghost2hugo") file, err := os.Open("shindakun-dot-net.ghost.2022-03-18-22-02-58.json") if err != nil { fmt.Println(err) } defer file.Close() b, err := io.ReadAll(file) if err != nil { fmt.Println(err) } var db GhostDatabase err = json.Unmarshal(b, &db) if err != nil { fmt.Println(err) } c := "`" + db.Db[0].Data.Posts[0].Mobiledoc + "`" un, err := strconv.Unquote(c) if err != nil { fmt.Println(err) } fmt.Printf("%v", un) var md Mobiledoc err = json.Unmarshal([]byte(un), &md) if err != nil { fmt.Println(err) } fmt.Printf("%#v", md) card := md.Cards[0][1] fmt.Printf("
card: %#v
", card) bbb := card.(map[string]interface{}) fmt.Println(bbb["markdown"])}
Original Link: https://dev.to/shindakun/attempting-to-learn-go-ghost-to-hugo-2-3ei1
Dev To
An online community for sharing and discovering great ideas, having debates, and making friendsMore About this Source Visit Dev To