17 Commits

Author	SHA1	Message	Date
Lunny Xiao	bd820aa9c5	Add context cache as a request level cache (#22294) ... To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.	2023-02-15 21:37:34 +08:00
KN4CK3R	e8186f1c0f	Map OIDC groups to Orgs/Teams (#21441) ... Fixes #19555 Test-Instructions: https://github.com/go-gitea/gitea/pull/21441#issuecomment-1419438000 This PR implements the mapping of user groups provided by OIDC providers to orgs teams in Gitea. The main part is a refactoring of the existing LDAP code to make it usable from different providers. Refactorings: - Moved the router auth code from module to service because of import cycles - Changed some model methods to take a `Context` parameter - Moved the mapping code from LDAP to a common location I've tested it with Keycloak but other providers should work too. The JSON mapping format is the same as for LDAP.  --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>	2023-02-08 14:44:42 +08:00
Gusted	1b53a9e914	Don't return duplicated users who can create org repo (#22560) ... - Currently the function `GetUsersWhoCanCreateOrgRepo` uses a query that is able to have duplicated users in the result, this is can happen under the condition that a user is in team that either is the owner team or has permission to create organization repositories. - Add test code to simulate the above condition for user 3, [`TestGetUsersWhoCanCreateOrgRepo`](a1fcb1cfb8/models/organization/org_test.go (L435)) is the test function that tests for this. - The fix is quite trivial use a map keyed by user id in order to drop duplicates. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>	2023-01-30 18:12:45 +08:00
Jason Song	659055138b	Secrets storage with SecretKey encrypted (#22142) ... Fork of #14483, but [gave up MasterKey](https://github.com/go-gitea/gitea/pull/14483#issuecomment-1350728557), and fixed some problems. Close #12065. Needed by #13539. Featrues: - Secrets for repo and org, not user yet. - Use SecretKey to encrypte/encrypt secrets. - Trim spaces of secret value. - Add a new locale ini block, to make it easy to support secrets for user. Snapshots: Repo level secrets:  Rrg level secrets  Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: KN4CK3R <admin@oldschoolhack.me>	2022-12-20 17:07:13 +08:00
Lunny Xiao	68704532c2	Rename almost all Ctx functions (#22071)	2022-12-10 10:46:31 +08:00
Lunny Xiao	0a7d3ff786	refactor some functions to support ctx as first parameter (#21878) ... Co-authored-by: KN4CK3R <admin@oldschoolhack.me> Co-authored-by: Lauris BH <lauris@nix.lv>	2022-12-03 10:48:26 +08:00
flynnnnnnnnnn	e81ccc406b	Implement FSFE REUSE for golang files (#21840) ... Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>	2022-11-27 18:20:29 +00:00
KN4CK3R	88d5275614	Do not allow Ghost access to limited visible user/org (#21849) ... The Ghost user should not be allowed to have access to a limited visible user/org. Co-authored-by: Lauris BH <lauris@nix.lv>	2022-11-17 14:29:33 -05:00
Lunny Xiao	34283a74e8	Allow detect whether it's in a database transaction for a context.Context (#21756) ... Fix #19513 This PR introduce a new db method `InTransaction(context.Context)`, and also builtin check on `db.TxContext` and `db.WithTx`. There is also a new method `db.AutoTx` has been introduced but could be used by other PRs. `WithTx` will always open a new transaction, if a transaction exist in context, return an error. `AutoTx` will try to open a new transaction if no transaction exist in context. That means it will always enter a transaction if there is no error. Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: 6543 <6543@obermui.de>	2022-11-12 21:18:50 +01:00
delvh	0ebb45cfe7	Replace all instances of fmt.Errorf(%v) with fmt.Errorf(%w) (#21551) ... Found using `find . -type f -name '*.go' -print -exec vim {} -c ':%s/fmt\.Errorf(\(.*\)%v\(.*\)err/fmt.Errorf(\1%w\2err/g' -c ':wq' \;` Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Andrew Thornton <art27@cantab.net> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>	2022-10-24 20:29:17 +01:00
KN4CK3R	c3b2e44392	Add team member invite by email (#20307) ... Allows to add (not registered) team members by email. related #5353 Invite by mail:  Pending invitations:  Email:  Join form:  Co-authored-by: Jack Hay <jjphay@gmail.com>	2022-10-19 14:40:28 +02:00
zeripath	716fcfcf72	Make every not exist error unwrappable to a fs.ErrNotExist (#20891) ... A lot of our code is repeatedly testing if individual errors are specific types of Not Exist errors. This is repetitative and unnecesary. `Unwrap() error` provides a common way of labelling an error as a NotExist error and we can/should use this. This PR has chosen to use the common `io/fs` errors e.g. `fs.ErrNotExist` for our errors. This is in some ways not completely correct as these are not filesystem errors but it seems like a reasonable thing to do and would allow us to simplify a lot of our code to `errors.Is(err, fs.ErrNotExist)` instead of `package.IsErr...NotExist(err)` I am open to suggestions to use a different base error - perhaps `models/db.ErrNotExist` if that would be felt to be better. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: delvh <dev.lh@web.de>	2022-10-18 07:50:37 +02:00
Lunny Xiao	0649c54275	Adjust transaction handling via db.Context (#20031)	2022-06-20 14:38:58 +02:00
Lunny Xiao	fd7d83ace6	Move almost all functions' parameter db.Engine to context.Context (#19748) ... * Move almost all functions' parameter db.Engine to context.Context * remove some unnecessary wrap functions	2022-05-20 22:08:52 +08:00
zuzuviewer	77e9346342	Fix duplicate entry error when add team member (#19702)	2022-05-14 16:30:19 +03:00
6543	06e4687cec	more context for models (#19511) ... make more usage of context, to have more db transaction in one session (make diff of #9307 smaller)	2022-04-28 13:48:48 +02:00
Lunny Xiao	b06b9a056c	Move organization related structs into sub package (#18518) ... * Move organization related structs into sub package * Fix test * Fix lint * Move more functions into sub packages * Fix bug * Fix test * Update models/organization/team_repo.go Co-authored-by: KN4CK3R <admin@oldschoolhack.me> * Apply suggestions from code review Co-authored-by: KN4CK3R <admin@oldschoolhack.me> * Fix fmt * Follow suggestion from @Gusted * Fix test * Fix test * Fix bug * Use ctx but db.DefaultContext on routers * Fix bug * Fix bug * fix bug * Update models/organization/team_user.go * Fix bug Co-authored-by: KN4CK3R <admin@oldschoolhack.me> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>	2022-03-29 14:29:02 +08:00