Jason Song
375fd15fbf
Refactor `modules/indexer` to make it more maintainable. And it can be
easier to support more features. I'm trying to solve some of issue
searching, this is a precursor to making functional changes.
Current supported engines and the index versions:
| engines | issues | code |
| - | - | - |
| db | Just a wrapper for database queries, doesn't need version | - |
| bleve | The version of index is **2** | The version of index is **6**
|
| elasticsearch | The old index has no version, will be treated as
version **0** in this PR | The version of index is **1** |
| meilisearch | The old index has no version, will be treated as version
**0** in this PR | - |
## Changes
### Split
Splited it into mutiple packages
```text
indexer
├── internal
│ ├── bleve
│ ├── db
│ ├── elasticsearch
│ └── meilisearch
├── code
│ ├── bleve
│ ├── elasticsearch
│ └── internal
└── issues
├── bleve
├── db
├── elasticsearch
├── internal
└── meilisearch
```
- `indexer/interanal`: Internal shared package for indexer.
- `indexer/interanal/[engine]`: Internal shared package for each engine
(bleve/db/elasticsearch/meilisearch).
- `indexer/code`: Implementations for code indexer.
- `indexer/code/internal`: Internal shared package for code indexer.
- `indexer/code/[engine]`: Implementation via each engine for code
indexer.
- `indexer/issues`: Implementations for issues indexer.
### Deduplication
- Combine `Init/Ping/Close` for code indexer and issues indexer.
- ~Combine `issues.indexerHolder` and `code.wrappedIndexer` to
`internal.IndexHolder`.~ Remove it, use dummy indexer instead when the
indexer is not ready.
- Duplicate two copies of creating ES clients.
- Duplicate two copies of `indexerID()`.
### Enhancement
- [x] Support index version for elasticsearch issues indexer, the old
index without version will be treated as version 0.
- [x] Fix spell of `elastic_search/ElasticSearch`, it should be
`Elasticsearch`.
- [x] Improve versioning of ES index. We don't need `Aliases`:
- Gitea does't need aliases for "Zero Downtime" because it never delete
old indexes.
- The old code of issues indexer uses the orignal name to create issue
index, so it's tricky to convert it to an alias.
- [x] Support index version for meilisearch issues indexer, the old
index without version will be treated as version 0.
- [x] Do "ping" only when `Ping` has been called, don't ping
periodically and cache the status.
- [x] Support the context parameter whenever possible.
- [x] Fix outdated example config.
- [x] Give up the requeue logic of issues indexer: When indexing fails,
call Ping to check if it was caused by the engine being unavailable, and
only requeue the task if the engine is unavailable.
- It is fragile and tricky, could cause data losing (It did happen when
I was doing some tests for this PR). And it works for ES only.
- Just always requeue the failed task, if it caused by bad data, it's a
bug of Gitea which should be fixed.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
|
||
|---|---|---|
| .devcontainer | ||
| .gitea | ||
| .github | ||
| assets | ||
| build | ||
| cmd | ||
| contrib | ||
| custom/conf | ||
| docker | ||
| docs | ||
| models | ||
| modules | ||
| options | ||
| public/img | ||
| routers | ||
| services | ||
| snap | ||
| templates | ||
| tests | ||
| web_src | ||
| .air.toml | ||
| .changelog.yml | ||
| .dockerignore | ||
| .drone.yml | ||
| .editorconfig | ||
| .eslintrc.yaml | ||
| .gitattributes | ||
| .gitignore | ||
| .gitpod.yml | ||
| .golangci.yml | ||
| .ignore | ||
| .markdownlint.yaml | ||
| .npmrc | ||
| .spectral.yaml | ||
| .stylelintrc.yaml | ||
| BSDmakefile | ||
| CHANGELOG.md | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| DCO | ||
| Dockerfile | ||
| Dockerfile.rootless | ||
| LICENSE | ||
| MAINTAINERS | ||
| Makefile | ||
| README.md | ||
| README_ZH.md | ||
| SECURITY.md | ||
| build.go | ||
| go.mod | ||
| go.sum | ||
| main.go | ||
| package-lock.json | ||
| package.json | ||
| playwright.config.js | ||
| poetry.lock | ||
| poetry.toml | ||
| pyproject.toml | ||
| vitest.config.js | ||
| webpack.config.js | ||
README.md
Gitea - Git with a cup of tea
Purpose
The goal of this project is to make the easiest, fastest, and most painless way of setting up a self-hosted Git service.
As Gitea is written in Go, it works across all the platforms and architectures that are supported by Go, including Linux, macOS, and Windows on x86, amd64, ARM and PowerPC architectures. You can try it out using the online demo. This project has been forked from Gogs since November of 2016, but a lot has changed.
Building
From the root of the source tree, run:
TAGS="bindata" make build
or if SQLite support is required:
TAGS="bindata sqlite sqlite_unlock_notify" make build
The build target is split into two sub-targets:
make backendwhich requires Go Stable, required version is defined in go.mod.make frontendwhich requires Node.js LTS or greater and Internet connectivity to download npm dependencies.
When building from the official source tarballs which include pre-built frontend files, the frontend target will not be triggered, making it possible to build without Node.js and Internet connectivity.
Parallelism (make -j <num>) is not supported.
More info: https://docs.gitea.io/en-us/install-from-source/
Using
./gitea web
NOTE: If you're interested in using our APIs, we have experimental support with documentation.
Contributing
Expected workflow is: Fork -> Patch -> Push -> Pull Request
NOTES:
- YOU MUST READ THE CONTRIBUTORS GUIDE BEFORE STARTING TO WORK ON A PULL REQUEST.
- If you have found a vulnerability in the project, please write privately to security@gitea.io. Thanks!
Translating
Translations are done through Crowdin. If you want to translate to a new language ask one of the managers in the Crowdin project to add a new language there.
You can also just create an issue for adding a language or ask on discord on the #translation channel. If you need context or find some translation issues, you can leave a comment on the string or ask on Discord. For general translation questions there is a section in the docs. Currently a bit empty but we hope to fill it as questions pop up.
https://docs.gitea.io/en-us/contributing/translation-guidelines/
Further information
For more information and instructions about how to install Gitea, please look at our documentation. If you have questions that are not covered by the documentation, you can get in contact with us on our Discord server or create a post in the discourse forum.
We maintain a list of Gitea-related projects at gitea/awesome-gitea.
The Hugo-based documentation theme is hosted at gitea/theme.
The official Gitea CLI is developed at gitea/tea.
Authors
Backers
Thank you to all our backers! 🙏 [Become a backer]
Sponsors
Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]
FAQ
How do you pronounce Gitea?
Gitea is pronounced /ɡɪ’ti:/ as in "gi-tea" with a hard g.
Why is this not hosted on a Gitea instance?
We're working on it.
License
This project is licensed under the MIT License. See the LICENSE file for the full license text.
Screenshots
Looking for an overview of the interface? Check it out!
 |
 |
 |
|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |