a6e9510c07
FIX blank internal pages ( #164 ) ( #292 )
...
Hello 👋
since it affected my deployment of the pages server I started to look into the problem of the blank pages and think I found a solution for it:
1. There is no check if the file response is empty, neither in cache retrieval nor in writing of a cache. Also the provided method for checking for empty responses had a bug.
2. I identified the redirect response to be the issue here. There is a cache write with the full cache key (e. g. rawContent/user/repo|branch|route/index.html) happening in the handling of the redirect response. But the written body here is empty. In the triggered request from the redirect response the server then finds a cache item to the key and serves the empty body. A quick fix is the check for empty file responses mentioned in 1.
3. The decision to redirect the user comes quite far down in the upstream function. Before that happens a lot of stuff that may not be important since after the redirect response comes a new request anyway. Also, I suspect that this causes the caching problem because there is a request to the forge server and its error handling with some recursions happening before. I propose to move two of the redirects before "Preparing"
4. The recursion in the upstream function makes it difficult to understand what is actually happening. I added some more logging to have an easier time with that.
5. I changed the default behaviour to append a trailing slash to the path to true. In my tested scenarios it happened anyway. This way there is no recursion happening before the redirect.
I am not developing in go frequently and rarely contribute to open source -> so feedback of all kind is appreciated
closes #164
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/292
Reviewed-by: 6543 <6543@obermui.de >
Reviewed-by: crapStone <codeberg@crapstone.dev >
Co-authored-by: Hoernschen <julian.hoernschemeyer@mailbox.org >
Co-committed-by: Hoernschen <julian.hoernschemeyer@mailbox.org >
2024-02-26 22:21:42 +00:00
7e80ade24b
Add config file and rework cli parsing and passing of config values ( #263 )
...
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/263
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: crapStone <me@crapstone.dev >
Co-committed-by: crapStone <me@crapstone.dev >
2024-02-15 16:08:29 +00:00
c1fbe861fe
rename gitea to forge in html error messages ( #287 )
...
closes #286
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/287
Reviewed-by: Andreas Shimokawa <ashimokawa@noreply.codeberg.org >
Co-authored-by: crapStone <crapstone01@gmail.com >
Co-committed-by: crapStone <crapstone01@gmail.com >
2024-02-11 12:43:25 +00:00
15916444e1
Fix panic in formatting not found message ( #276 )
...
Fix panic that was found in the logs, `targetOpt` is `nil`:
http2: panic serving 10.0.3.1:[...]: runtime error: invalid memory address or nil pointer dereference
net/http.(*http2serverConn).runHandler.func1()
/usr/local/go/src/net/http/h2_bundle.go:6104 +0x145
panic({0x19c6820, 0x2d66db0})
/usr/local/go/src/runtime/panic.go:884 +0x213
codeberg.org/codeberg/pages/server/handler.handleSubDomain({{0x2008c68, 0xc00047df90}, 0x2, {0x0, 0x0}, {0xc0fe3ef800, 0x55, 0x1f4}, {0xc00047dfa0, 0x1, ...}, ...}, ...)
/woodpecker/src/codeberg.org/Codeberg/pages-server/server/handler/handler_sub_domain.go:59 +0x5e0
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/276
Reviewed-by: crapStone <codeberg@crapstone.dev >
Co-authored-by: Gusted <postmaster@gusted.xyz >
Co-committed-by: Gusted <postmaster@gusted.xyz >
2024-01-18 14:31:46 +00:00
fffb8ffcb6
remove use of rawInfoPage redirect ( #261 )
...
closes #244
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/261
Co-authored-by: crapStone <crapstone01@gmail.com >
Co-committed-by: crapStone <crapstone01@gmail.com >
2023-11-16 17:33:39 +00:00
cbb2ce6d07
add go templating engine for error page and make errors more clear ( #260 )
...
ping #199
closes #213
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/260
Co-authored-by: crapStone <crapstone01@gmail.com >
Co-committed-by: crapStone <crapstone01@gmail.com >
2023-11-16 17:11:35 +00:00
1e1c67be93
let gitea client send user-agent with version ( #258 )
...
closes #255
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/258
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: crapStone <crapstone01@gmail.com >
Co-committed-by: crapStone <crapstone01@gmail.com >
2023-11-15 15:25:14 +00:00
a8272f0ce9
Don't send server version to client ( #254 )
...
closes #247
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/254
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: crapStone <crapstone01@gmail.com >
Co-committed-by: crapStone <crapstone01@gmail.com >
2023-11-15 01:49:29 +00:00
56d3e291c4
Security Fix: clean paths correctly to avoid circumvention of BlacklistedPaths
2023-08-27 10:13:15 +02:00
d720d25e42
Use http.NoBody as per linter ( #231 )
...
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/231
Reviewed-by: Gusted <gusted@noreply.codeberg.org >
2023-07-17 19:44:58 +00:00
7f318f89a6
Fix escaped error message ( #230 )
...
- This specific message will [already be generated](https://codeberg.org/Codeberg/pages-server/src/commit/974229681f4cc7f1ed31df9b05eabef2df01380/html/error.go#L44 ) when `http.StatusMisdirectedRequest` is set as status with [an empty message](974229681f/html/error.go (L25-L28)https://codeberg.org/Codeberg/pages-server/issues/228
Co-authored-by: Gusted <postmaster@gusted.xyz >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/230
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2023-07-16 22:34:46 +00:00
974229681f
Initial redirects implementation ( #148 )
...
Adds basic support for `_redirects` files. It supports a subset of what IPFS supports: https://docs.ipfs.tech/how-to/websites-on-ipfs/redirects-and-custom-404s/
Example:
```
/redirect https://example.com/ 301
/another-redirect /page 301
/302 https://example.com/ 302
/app/* /index.html 200
/articles/* /posts/:splat 301
```
301 redirect: https://video-prize-ranch.localhost.mock.directory:4430/redirect
SPA rewrite: https://video-prize-ranch.localhost.mock.directory:4430/app/path/path
Catch-all with splat: https://video-prize-ranch.localhost.mock.directory:4430/articles/path/path
Closes #46
Co-authored-by: video-prize-ranch <cb.8a3w5@simplelogin.co >
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/148
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: video-prize-ranch <video-prize-ranch@noreply.codeberg.org >
Co-committed-by: video-prize-ranch <video-prize-ranch@noreply.codeberg.org >
2023-03-30 21:36:31 +00:00
c40dddf471
Fix certificate renewal ( #209 )
...
A database bug in xorm.go prevents the pages-server from saving a
renewed certificate for a domain that already has one in the database.
Co-authored-by: crystal <crystal@noreply.codeberg.org >
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/209
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Crystal <crystal@noreply.codeberg.org >
Co-committed-by: Crystal <crystal@noreply.codeberg.org >
2023-03-20 22:57:26 +00:00
c9050e5722
Handle Relative Symlinks ( #205 )
...
enhance #114
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/205
2023-03-11 05:07:17 +00:00
42d5802b9b
Allow to define default branches ( #125 )
...
This try to address #115
Co-authored-by: Simon Vieille <simon@deblan.fr >
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/125
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: deblan <deblan@noreply.codeberg.org >
Co-committed-by: deblan <deblan@noreply.codeberg.org >
2023-02-14 03:03:00 +00:00
0adac9a5b1
fix http -> https redirect and add integration tests for it ( #184 )
...
and more logging
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/184
2023-02-14 02:23:28 +00:00
42b3f8d1b7
use mockery for mock code generation ( #185 )
...
close #181
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/185
2023-02-13 23:13:30 +00:00
9a3d1c36dc
Document more flags & make http port customizable ( #183 )
...
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/183
2023-02-13 20:14:45 +00:00
46316f9e2f
Fix raw domain for branches with custom domains and index.html ( #159 )
...
fix #156
fix #157
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/159
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: crystal <crystal@noreply.codeberg.org >
Co-committed-by: crystal <crystal@noreply.codeberg.org >
2023-02-11 03:12:42 +00:00
5753f7136d
Move acmeClient creation into own file & struct ( #179 )
...
get rid of gobal vars and make make functions with less args :)
tldr: collect funcs and create a own ACME client to manage that stuff
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/179
2023-02-11 02:29:08 +00:00
fd643d15f0
Drop: pogreb support ( #175 )
...
followup of #173
close #95
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/175
2023-02-11 02:04:57 +00:00
272c7ca76f
Fix xorm regressions by handle wildcard certs correctly ( #177 )
...
close #176
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/177
2023-02-11 01:26:21 +00:00
d8d119b0b3
Fix Cache Bug ( #178 )
...
error io.EOF is gracefully end of file read.
so we don't need to cancel cache saving
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/178
2023-02-11 00:31:56 +00:00
7b35a192bf
Add cert store option based on sqlite3, mysql & postgres ( #173 )
...
Deprecate **pogreb**!
close #169
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/173
2023-02-10 03:00:14 +00:00
7fce7cf68b
Added Whitespace Trimming TXT DNS Records ( #152 )
...
Solves https://codeberg.org/Codeberg/Community/issues/823 and https://codeberg.org/Codeberg/pages-server/issues/143
Co-authored-by: foehammer127 <foehammer127@gmail.com >
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/152
Reviewed-by: Otto <otto@codeberg.org >
Reviewed-by: Gusted <gusted@noreply.codeberg.org >
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: foehammer <foehammer@noreply.codeberg.org >
Co-committed-by: foehammer <foehammer@noreply.codeberg.org >
2023-02-10 01:44:44 +00:00
8b1f497bc4
Allow to use certificate even if domain validation fails ( #160 )
...
- Currently if the canonical domain validations fails(either for
legitimate reasons or for bug reasons like the request to Gitea/Forgejo
failing) it will use main domain certificate, which in the case for
custom domains will warrant a security error as the certificate isn't
issued to the custom domain.
- This patch handles this situation more gracefully and instead only
disallow obtaining a certificate if the domain validation fails, so in
the case that a certificate still exists it can still be used even if
the canonical domain validation fails. There's a small side effect,
legitimate users that remove domains from `.domain` will still be able
to use the removed domain(as long as the DNS records exists) as long as
the certificate currently hold by pages-server isn't expired.
- Given the increased usage in custom domains that are resulting in
errors, I think it ways more than the side effect.
- In order to future-proof against future slowdowns of instances, add a retry mechanism to the domain validation function, such that it's more likely to succeed even if the instance is not responding.
- Refactor the code a bit and add some comments.
Co-authored-by: Gusted <postmaster@gusted.xyz >
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/160
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2023-02-10 01:38:15 +00:00
513e79832a
Use correct log level for CheckCanonicalDomain ( #162 )
...
- Currently any error generated by requesting the `.domains` file of a repository would be logged under the info log level, which isn't the correct log level when we exclude the not found error.
- Use warn log level if the error isn't the not found error.
Co-authored-by: Gusted <postmaster@gusted.xyz >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/162
Reviewed-by: Otto <otto@codeberg.org >
2023-01-22 18:52:21 +00:00
bd538abd37
Fix wrong redirect on custom domain with path ( #154 )
...
closes #153
Co-authored-by: crapStone <crapstone01@gmail.com >
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/154
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: crapStone <crapstone@noreply.codeberg.org >
Co-committed-by: crapStone <crapstone@noreply.codeberg.org >
2023-01-11 00:00:37 +00:00
c286b3b1d0
Added TokenBucket to limit the rate of validation failures ( #151 )
...
Added new TockenBucket named `acmeClientFailLimit` to avoid being banned because of the [Failed validation limit](https://letsencrypt.org/docs/failed-validation-limit/ ) of Let's Encrypt.
The behaviour is similar to the other limiters blocking the `obtainCert` func ensuring rate under limit.
Co-authored-by: fsologureng <sologuren@estudiohum.cl >
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/151
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Felipe Leopoldo Sologuren Gutiérrez <fsologureng@noreply.codeberg.org >
Co-committed-by: Felipe Leopoldo Sologuren Gutiérrez <fsologureng@noreply.codeberg.org >
2023-01-04 05:26:14 +00:00
98d198d419
Safely get certificate's leaf ( #150 )
...
- It's not guaranteed that `tls.X509KeyPair` will set `c.Leaf`.
- This patch fixes this by using a wrapper that parses the leaf
certificate(in bytes) if `c.Leaf` wasn't set.
- Resolves #149
Co-authored-by: Gusted <postmaster@gusted.xyz >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/150
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2023-01-04 04:51:27 +00:00
caeb1a4acb
Return a 404 if there is no repository ( #141 )
...
If no repository is found the user expects a 404 status code
instead of a dependency failed status code (as it was before).
Signed-off-by: Jan Klippel <c0d3b3rg@kl1pp3l.de >
Fixes: https://codeberg.org/Codeberg/Community/issues/809
Co-authored-by: Jan Klippel <c0d3b3rg@kl1pp3l.de >
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/141
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: jklippel <jklippel@noreply.codeberg.org >
Co-committed-by: jklippel <jklippel@noreply.codeberg.org >
2022-11-22 21:26:10 +00:00
f2f943c0d8
Remove unnecessary conversion ( #139 )
...
- Remove unnecessary type conversion.
- Enforce via CI
Co-authored-by: Gusted <williamzijl7@hotmail.com >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/139
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2022-11-15 16:15:11 +01:00
3c61a39864
Enable http/2 support ( #137 )
...
As per [the documentation](https://pkg.go.dev/net/http#Serve ), it doesn't enable HTTP2 by-default, unless we enable it via the `NextProtos` option.
Co-authored-by: Gusted <williamzijl7@hotmail.com >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/137
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2022-11-12 22:25:20 +01:00
4565481643
refactor: finish use default const for defaultPagesBranch and defaultPagesRepo
2022-11-12 21:16:11 +01:00
c827a28dd8
defaultPagesBranch
2022-11-12 21:13:13 +01:00
aa90356f0a
use a const for defaultPagesRepo
2022-11-12 21:10:16 +01:00
b6d0a04b21
refactor: rm not needed type conversion
2022-11-12 21:04:34 +01:00
6c63b66ce4
Refactor split long functions ( #135 )
...
we have big functions that handle all stuff ... we should split this into smaler chuncks so we could test them seperate and make clear cuts in what happens where
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/135
2022-11-12 20:43:44 +01:00
b9966487f6
switch to std http implementation instead of fasthttp ( #106 )
...
close #100
close #109
close #113
close #28
close #63
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/106
2022-11-12 20:37:20 +01:00
91b54bef29
add newline
2022-11-07 23:09:41 +01:00
bf9a08e1fd
Fatal on ACME Client creation failure ( #133 )
...
- For production(*cough* Codeberg *cough*), it's important to not use
mock certs. So fail right from the start if this is the case and not try
to "handle it gracefully", as it would break production.
- Resolves #131
CC @6543
Co-authored-by: Gusted <williamzijl7@hotmail.com >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/133
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2022-11-07 16:27:37 +01:00
091e6c8ed9
Add explicit logging in GetBranchTimestamp ( #130 )
...
- Logs are currently indicating that it's returning `nil` in valid
scenarios, therefor this patch adds extra logging in this code to
better understand what it is doing in this function.
Co-authored-by: Gusted <williamzijl7@hotmail.com >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/130
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2022-09-18 16:13:27 +02:00
8f2699407d
Make verbose checks in tryBranch ( #127 )
...
- It's likely that the tryBranch is returning false when it should be returning true, make these logs more verbose so they show up on production logs.
Co-authored-by: Gusted <williamzijl7@hotmail.com >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/127
Reviewed-by: 6543 <6543@obermui.de >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2022-09-13 23:06:31 +02:00
1ae50735a1
Add host to handler logging ( #123 )
...
- Add the host to the Handler's logging fields, so you don't just see the path, but also which domain was being requested.
Co-authored-by: Gusted <williamzijl7@hotmail.com >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/123
Reviewed-by: 6543 <6543@noreply.codeberg.org >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2022-08-13 18:03:31 +02:00
dc41a4caf4
Add Support to Follow Symlinks and LFS ( #114 )
...
close #79
close #80
close #91
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/114
2022-08-12 06:40:12 +02:00
876a53d9a2
Improve logging ( #116 )
...
- Actually log useful information at their respective log level.
- Add logs in hot-paths to be able to deep-dive and debug specific requests (see server/handler.go)
- Add more information to existing fields(e.g. the host that the user is visiting, this was noted by @fnetX).
Co-authored-by: Gusted <williamzijl7@hotmail.com >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/116
Reviewed-by: 6543 <6543@noreply.codeberg.org >
Co-authored-by: Gusted <gusted@noreply.codeberg.org >
Co-committed-by: Gusted <gusted@noreply.codeberg.org >
2022-08-12 05:06:26 +02:00
e06900d5e5
fix lint issue
2022-08-08 15:25:31 +02:00
8207586a48
just fix bcaceda711
2022-07-15 21:39:42 +02:00
bcaceda711
dont cache if ContentLength greater fileCacheSizeLimit ( #108 )
...
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/108
Reviewed-by: Otto <otto@codeberg.org >
2022-07-15 21:21:26 +02:00
5411c96ef3
Tell fasthttp to not set "Content-Length: 0" on non cached content ( #107 )
...
fix #97
Co-authored-by: 6543 <6543@obermui.de >
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/107
2022-07-15 21:06:05 +02:00
