An Interest In:
Web News this Week
- March 21, 2024
- March 20, 2024
- March 19, 2024
- March 18, 2024
- March 17, 2024
- March 16, 2024
- March 15, 2024
My Contribution to a Popular Open-Source Package Caused a Panic in Golang Projects
Hello, today I will tell you how my development to a popular open-source project caused errors to people using the project around the world and what I learned from this process.
Content
Swaggo Project
Our Requirement and My Contribution
The Problem
Reason of The Problem
Function Definition Withtout Body (go:linkname)
The Solution
Learned Lessons
SWAGGO Project
Creating swagger documents for your API projects developed in Go language is a little different from languages such as Java, C#. Generally, you can create it by adding Swagger directives in the comment lines of the code you write, and you don't have many alternatives for this.
The swaggo/swag project is one of the most popular and widely used solutions.
The basic process of this project is creating the swagger document by browsing the Go files and processing the comment lines in the definitions. It does this by creating an Abstract Syntax Tree (AST) and navigating it.
Our Requirements and My Contribution
While trying to create a swagger document for a project with my teammate, we realized that we could not make the definition we wanted using the swaggo/swag tool.
What we wanted to do was to define the request and response objects inside the handler function we created and ensure that they can only be accessed from within the function. But when we did this, the swaggo/swag parser package could not access the related request and response objects.
I wanted to make a contribution to the swaggo/swag library in order to overcome this problem.
First of all, I opened an issue to the library and asked if we needed such a thing and that I wanted to contribute by making this development.
In response, they said that it is not possible to parse private definitions and when they asked how I would do it, they were curious about my proposal and were waiting to see it.
I rolled up my sleeves. I did the development at the first opportunity and opened PR for the project.
As a result of the development I made, request/response objects became definable within the function.
add functio scoped struct parse, tests #1283
Describe the PRI added the functionality of parsing function scoped structs.
With this development we will be able to define our request response types in the same function scope as our handlers:
package main// @Param request body main.Fun.request true "query params" // @Success 200 {object} main.Fun.response// @Router /test [post]func Fun() { type request struct { Name string } type response struct { Name string Child string }}
Relation issuehttps://github.com/swaggo/swag/issues/1274
Additional contextAll tests passed in parser_tests.go. Changes look backward compatible.
The Problem
It's a Friday evening... The work is over, there are minutes... I'm looking forward to the weekend... And... I got a message from my teammate that one of our projects got an error
I actually want to share the full conversation:
Translation:
- Vural: bro I will tell you something
- Me: yes bro
- Vural: one of our services getting error while generating swagger do, and guess what on which part :D
- Me: is that because of my development asdfasdf
- Vural: yes, I guess asdfsadfs
- Me: you can't be serious asdfasdf
We immediately went to the project's Github page and looked at the issues. And what we saw made me a little uneasy.
Many swaggo/swag package users were experiencing errors and they were all getting errors in the CI/CD pipelines or while manually generating swagger due to my development.
Segmentation fault running `swag init` after 1.8.5 release #1309
Describe the bugSince 1.8.5 was released, when running swag init --parseDependency true
on my code I encounter a segfault. After switching back to 1.8.4 (or other versions in the 1.8 line) the command completes successfully.
To ReproduceI'm unable to share the entire code this happens on. I will see if I can isolate this to a small code sample I can share.
Stack trace
panic: runtime error: invalid memory address or nil pointer dereference[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x85c8a8]goroutine 1 [running]:github.com/swaggo/swag.(*PackagesDefinitions).parseFunctionScopedTypesFromFile(0xc00000e7c8, 0xc00035b980, {0xc0003e15c0, 0x18}, 0xc000cff680) /home/sean/go/pkg/mod/github.com/swaggo/[email protected]/packages.go:168 +0xa8github.com/swaggo/swag.(*PackagesDefinitions).ParseTypes(0xc00000e7c8) /home/sean/go/pkg/mod/github.com/swaggo/[email protected]/packages.go:110 +0xc9github.com/swaggo/swag.(*Parser).ParseAPIMultiSearchDir(0xc0001e22a0, {0xc0001d7fb0?, 0x1?, 0x0?}, {0x9587e8?, 0x7?}, 0x64) /home/sean/go/pkg/mod/github.com/swaggo/[email protected]/parser.go:362 +0x3bfgithub.com/swaggo/swag/gen.(*Gen).Build(0xc0001db980, 0xc0001e1790) /home/sean/go/pkg/mod/github.com/swaggo/[email protected]/gen/gen.go:177 +0x5c9main.initAction(0xc0001af680?) /home/sean/go/pkg/mod/github.com/swaggo/[email protected]/cmd/swag/main.go:151 +0x757github.com/urfave/cli/v2.(*Command).Run(0xc0001ad560, 0xc0001ec580) /home/sean/go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:163 +0x5bbgithub.com/urfave/cli/v2.(*App).RunContext(0xc000204000, {0xa25860?, 0xc0000240f0}, {0xc000020080, 0x4, 0x4}) /home/sean/go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:313 +0xb48github.com/urfave/cli/v2.(*App).Run(...) /home/sean/go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:224main.main() /home/sean/go/pkg/mod/github.com/swaggo/[email protected]/cmd/swag/main.go:221 +0x55d
Your swag version1.8.5 (1.8.4 and earlier worked)
Go version: 1.18
Desktop (please complete the following information):
- OS: Pop! OS 22.04 (essentially Ubuntu)
Frankly, I was surprised by this situation because I wrote unit tests for my development and the code was working correctly. In fact, the successful passing of the current unit tests proved that it did not break any previous development.
So what could be the problem?
Reason of The Problem
I'm attaching a screenshot from my development for swaggo/swag.
As we can see in the error messages, the error is received in the parseFunctionScopedTypesFromFile
method. In fact, the exact line number of the error is 168.
Now let's take a closer look at this method I wrote.
In line 168, AST is visited and if there is FunctionDecleration among the definitions, what written inside the defined function is iterated with a for loop.
So if we are sure that a function definition exists, why do we get an error when trying to iterate this function definition with a for loop?
What causes the nil pointer error?
When we look at the functionDeclaration.Body
variable, we see that it is a pointer type.
I didn't do a nil check here because I assumed that if a function was defined, that function would have a body definition.
But in reality it may not be so.
Function Definition Without Body
Does this term sound foreign to you? Can we define a function without writing a body?
Consider the code below, do you think this code is a valid definition in Go?
Will we get an error if we try to run it?
Yes, as you guessed, we cannot run or even compile this code because it is not a valid definition.
Can we run the code below?
Here, unlike the previous one, the answer will be yes, we can run it.
A little-known feature of Go is that we can link a function definition to the function definitions of other packages with the go:linkname
directive.
Thus, we can define a function without defining a body.
And if you have noticed, this feature allowed me to use a private function defined under the Go runtime package.
External dependencies that we use in our projects can have such function definitions. And when we tried to create a swagger document with the swag --parseDependency
command, functions with this definition caused a panic in my development because it also parsed dependencies.
The Solution
I was going to do a development to fix the problem right away, but the good thing about the open-source community is that another developer wrote the code that fixed the bug before me and created a PR.
fix: funcDeclaration body check #1310
Describe the PRNil pointer check in function scoped types.
Relation issue#1309
This is fixes problem on running swag build.
Learned Lessons
With this problem, I learned some lessons for myself.
Never bypass the nil pointer check hypothetically
Unit tests may not always be enough. Do the end to end test
We can write functions without body definition using go:linkname
with Go, we can import functions from private packages
Never make the package dependencies the latest version, specify the version you are sure is working
I hope it was a fun and useful content for you. Take care of yourself, I wish you bug-free codes
Let's Connect
You can support me on Github: Support mstrYoda on Github
Original Link: https://dev.to/mstryoda/my-contribution-to-a-popular-open-source-package-caused-a-panic-in-golang-projects-14pi
Dev To
An online community for sharing and discovering great ideas, having debates, and making friendsMore About this Source Visit Dev To