Donovan Daniels
5edbb507b6
[UploadPresenter] Normalize tags & strip metatags
2024-04-05 23:29:19 -05:00
Earlopain
e3fdc5d61b
[Pools] Fix discrepancy between index page count and actual page count
...
One was using `user.per_page`, the other config.posts_per_page. Consolidate this
logic to `paginate_posts` to make it more obvious what's going on.
2024-04-04 22:52:49 +02:00
Donovan Daniels
42c59ac384
[BUR] Move "duplicate of" text location
2024-03-25 19:53:43 -05:00
Earlopain
e4799d9f83
[Prod] Dump NewRelic
...
Need to look into alternatives
2024-03-23 20:52:31 +01:00
Earlopain
853790272b
[Posts] Allow excluding multiple ids in search
2024-03-03 20:04:02 +01:00
Earlopain
17b99821e3
[Posts] Fix searches for huge ids erroring
...
This is because they can exceed Integer.MAX_VALUE on the opensearch side
2024-02-28 17:25:54 +01:00
Earlopain
fc7d84affd
[RuboCop] Enable Style/FrozenStringLiteralComment
...
This reduces allocations on the posts page by about 5%, from basic testing
2024-02-25 18:15:55 +01:00
Earlopain
bc06f8cea2
[Misc] Fix all remaining (hopefully) issues with frozen string literals
...
+"" is a bit of a copout, that could be done better but very easy to do in these more complicated cases
2024-02-25 16:41:35 +01:00
Earlopain
930b4b905c
[Pools] Fix show/gallery for frozen string literals
...
Get rid of the presenters/post sets.
2024-02-25 11:38:55 +01:00
Earlopain
0381653e80
[Pools] Use paginate method for posts
2024-02-24 19:57:29 +01:00
Earlopain
56e849695b
[Cleanup] Remove readonly only
...
Since its introduction 3 years ago, it got used once I believe
I can't even say if this properly works anymore.
If something like that is actually desired in the future, it should just be a whitelist instead
2024-02-23 16:48:56 +01:00
Earlopain
b18e921174
[Posts] Show a nicer error message on upload if redis is down
2024-02-19 16:32:58 +01:00
Earlopain
c9fa546fb3
[Cleanup] Remove deny_transitives from tag alias approve!
...
Passing true would error because the error class doesn't exist.
BURs check this condition right before.
2024-02-17 12:12:39 +01:00
Earlopain
192fb2c1ed
[Discord] Fix counts for AIBUR report
2024-02-15 17:00:31 +01:00
Earlopain
1a7df4a497
[Misc] Use libvips provided color profile
2024-02-04 16:42:43 +01:00
Earlopain
ae96db548e
[Cleanup] Avoid nested methods
...
These are redefined every time the enclosing method is called
2024-01-21 17:55:11 +01:00
Earlopain
dd7106ed2d
[Discord] Add a report for AIBURs
2024-01-21 17:28:46 +01:00
Earlopain
8e94b9776e
[Discord] Generalize reports a bit more
2024-01-14 13:39:57 +01:00
Earlopain
cd188a797f
[Tests] Handle empty git hash
...
The app directory isn't marked as safe => a test fails for accessing nil
Not sure how there's only 1 test failing because of this.
Either way, this is more correct
2024-01-12 22:56:20 +00:00
Earlopain
40792f6d43
[Discord] Add moderator report
2024-01-09 17:24:48 +01:00
Earlopain
3e1af2183a
[Discord] Move some things for discord webhook to base class
2024-01-09 17:12:43 +01:00
Earlopain
d9bfd29b48
[Uploads] Fix missing paren in error message
2023-12-04 20:56:53 +01:00
Earlopain
6bb4c272cc
[Gems] Bump rubocop
2023-12-03 15:36:37 +01:00
Earlopain
f8098721ec
[Misc] Remove memoist gem dependency
...
The author has passed away and some functionality is broken with recent
ruby versions already. We only use a subset of it, so it's easy to replace.
Some places:
* Didn't even use it
* Used it but the method was already cached through memcached
* Just don't need it
Some places I question if it is needed but I'll just leave them there
since I don't want to benchmark
2023-11-08 22:21:11 +01:00
Earlopain
8cd56b5cd2
[Posts] Prevent extra post lookup for seq/prev nav buttons
...
The tag search already returns a post instance. There's no point in
extracting the id only to look it up again later.
2023-11-08 19:22:01 +01:00
Earlopain
7dcaf81979
[Uploads] Don't follow redirects to non-whitelisted urls
...
Also improves the existing tests a bit by being checking the error message
2023-10-20 21:00:32 +02:00
Earlopain
776866b873
[Cleanup] Move blocked ip checking out of config
...
These three used methods do basically the same, except more correct.
fc00::/7 are considered private while here only fd00::/8 was checked
2023-10-20 19:58:10 +02:00
Earlopain
cc6e18797a
[Uploads] Correctly follow redirects
...
Some sites serve urls that redirect by default. This would append the
"You are being redirected" html to the file being downloaded after that
and subsequently report that text/html is not a valid format
2023-10-15 13:52:38 +02:00
Earlopain
2620ea983e
[Wiki] Better diffing view
2023-10-15 13:08:28 +02:00
Earlopain
d16b7b0b40
[Prod] Limit the amount of data sent to NewRelic
2023-10-11 19:45:11 +02:00
Earlopain
e3256bb313
[Prod] Add instrumentation to login attempts
2023-10-10 17:00:59 +02:00
Earlopain
728a8caa60
[Prod] Rework NewRelic logging
...
* Use config to log request params
* Stop logging the session. There's nothing useful there
* Stop logging a users level
2023-10-10 16:23:43 +02:00
Earlopain
3a58ac0938
Switch to OpenSearch ( #550 )
...
Run the following commands to import data into OpenSearch:
Post.document_store.create_index!
PostVersion.document_store.create_index!
Post.document_store.import
PostVersion.document_store.import
* Add opensearch client
* Add url param to force use opensearch
* Switch import method over to opensearch
* Index to opensearch as well
* Add option to gradually roll out os
* Index os with separate queue
* Move os post creation to job as well
* Exclusively use the OpenSearch client
* Stop enqueuing OsIndexUpdateJob
* Remove remaining elasticsearch code
Bump faraday since v2 is no longer blocked
2023-10-02 18:57:07 +02:00
Earlopain
f7b829ba34
[Elasticsearch] Perform create callback inline
...
I changed this previously because I didn't see the reason behind the distinction.
When the job queues are full for example because of approvals of large BURs
it's desirable for newly uploaded posts to still appear in search results.
2023-09-18 20:28:53 +02:00
Earlopain
b6dc473a55
[Elasticsearch] Bring back the proxy object
2023-09-16 17:09:33 +02:00
Earlopain
ddcb7c1af1
[Elasticsearch] Implement index_name
2023-09-16 15:39:49 +02:00
Earlopain
c65288d2cd
[Elasticsearch] Move PaginatedArray to own file
2023-09-16 15:36:10 +02:00
Earlopain
b4e16fa0fd
[Elasticsearch] Implement Response
2023-09-16 15:30:28 +02:00
Earlopain
be71bf7a16
[Elasticsearch] Move implementation to own folder
2023-09-16 14:29:04 +02:00
Earlopain
f1d5864724
[Elasticsearch] Move more code out of indexable
2023-09-16 14:26:04 +02:00
Earlopain
d41496d254
[Elasticsearch] Implement search
2023-09-16 13:40:50 +02:00
Earlopain
a8b085e714
[Elasticsearch] Implement delete_document
2023-09-16 13:23:25 +02:00
Earlopain
123a96bd26
[Elasticsearch] Implement index_document
...
Also makes the callbacks for update the same as for create.
I don't see why creation should run inline, same for reownering.
2023-09-16 13:19:21 +02:00
Earlopain
3285d546f9
[Elasticsearch] Move index creation to DocumentStore
2023-09-14 21:53:26 +02:00
Earlopain
855388b092
[Elasticsearch] Remove a few more easy usages of __elasticsearch__
2023-09-14 20:56:56 +02:00
Earlopain
a79e25dfbd
[Elasticsearch] Start implementing own elasticsearch-model
...
This implements create_index!
2023-09-14 20:49:19 +02:00
Earlopain
056ce3e7aa
[Cleanup] Remove without_safe_mode from CurrentUser
...
It really did two things and was primarily intented for automated actions
so that all posts get matched regardless of rating and status.
Now there's a dedicated method to do just that
2023-09-11 18:30:25 +02:00
Donovan Daniels
115c140cc2
[Posts] Make comment_disabled not hide comments ( #546 )
...
* [Posts] Make comment_disabled not hide comments
* [VoteManager] Disallow voting on locked/disabled comments
* [Comments] Disable editing/hiding disabled & locked comments
* Tweaking
* Allow hiding when comments locked
* Don't allow mods to vote when comments locked
* Change wording from disabled to locked everywhere
---------
Co-authored-by: Earlopain <14981592+Earlopain@users.noreply.github.com>
2023-09-05 17:27:05 +02:00
Earlopain
d2f8943b63
[TagQuery] Simplify order metatags special handling
...
`rank` can be moved inside case/when and the rest is not needed since
that field is not nullable
2023-09-05 16:10:32 +02:00
Earlopain
1b9ea498d3
[TagQuery] Move more stuff into the base elastic class
2023-09-05 16:03:13 +02:00
Earlopain
636937a118
[TagQuery] Move generic elastic code to base class
2023-09-05 15:35:23 +02:00
Earlopain
2290d7045e
[TagQuery] Support any/none for ORed metatags
...
Turns out it's really simple
2023-09-03 21:25:44 +02:00
Earlopain
6e473193fe
[TagQuery] Fix exception for user_id metatag
2023-09-03 20:37:44 +02:00
Earlopain
17e9c052cd
[Ads] Support keyword targets
...
Basically a redo of b61856af6d0842a50ad5
2023-09-03 18:44:42 +02:00
Earlopain
9203dcdb82
[Posts] Add a method to get tags that a post is tagged with
...
Better performance than the regex as well
2023-09-03 18:24:40 +02:00
Earlopain
9d2a4d6996
[TagQuery] Support ~ for metatags
2023-09-03 17:02:07 +02:00
Earlopain
6b3b24daa2
[TagQuery] Use consistent naming for neg/must_not
2023-09-03 16:43:10 +02:00
Earlopain
040d6454b7
[TagQuery] Remove nesting from elastic query
...
Tags got their own bool query. There doesn't seem to be a reason for that.
Metatags have the same search meaning as normal tags
2023-09-03 15:40:48 +02:00
Earlopain
177558b5d5
[TagQuery] Use consistent naming for tag fields
2023-09-03 15:23:27 +02:00
Earlopain
b2c7b338c0
[TagQuery] Improve description search
...
These all works:
* description:test
* description:"multiple words followed by each other"
* description:"multiple words followed by each oth*"
2023-09-03 15:12:40 +02:00
Earlopain
be75cf3349
[QueryBuilder] Fix searching by tag counts
2023-08-31 08:57:47 +02:00
Earlopain
b65741456f
[QueryBuilder] Move count validation into the query class
2023-08-24 13:43:01 +02:00
Earlopain
c8d7bebd72
[QueryBuilder] Move parsing code into own class
...
Introduces the concept of "free" tags to account for automatically added tags.
Introduce `resolve_aliases` which can be used instead of `raw_tag_match`
2023-08-24 13:16:44 +02:00
Earlopain
9848c18ace
[QueryBuilder] Move value parsing into separate module
...
Also add some tests
2023-08-23 14:18:51 +02:00
Earlopain
77fd4e5bb7
[Tags] Rework and rename has_metatag?`
...
The function returns the metatag value, it shouldn't be named that way.
The new methods are more explicit and also more performant.
2023-08-22 13:57:41 +02:00
Earlopain
f8529bc60d
[Cleanup] Remove unused format parameter from post set
2023-08-20 18:21:41 +02:00
Earlopain
31957d5dbc
[QueryBuilder] Support negation/multiple values for ranges
2023-08-20 14:29:20 +02:00
Earlopain
ba3030a14c
[QueryBuilder] Create seperate method for creating the range term
2023-08-20 12:58:30 +02:00
Earlopain
da86cc3ae6
[QueryBuilder] Don't pass q around everywhere
2023-08-20 12:19:05 +02:00
Earlopain
c3d9a5c28e
[QueryBuilder] Convert source and delreason to add_to_query
2023-08-20 12:05:45 +02:00
Earlopain
19a74bc2e3
[QueryBuilder] Remove back-and-forth conversion with sql/elastic
2023-08-20 11:43:36 +02:00
Earlopain
df4112d720
[QueryBuilder] Support multiple metatags for a bunch of cases
2023-08-19 22:43:12 +02:00
Earlopain
23eddc4a5a
[QueryBuilder] Allow multiple lock types in single search query
2023-08-19 21:11:26 +02:00
Earlopain
e5c64583c7
[QueryBuilder] Add method for searching normal field
2023-08-19 20:39:15 +02:00
Earlopain
2f56d60cb5
[QueryBuilder] Simply parsing of rating metatag
2023-08-19 18:52:29 +02:00
Earlopain
f193e159eb
[QueryBuilder] Introduce methods to handle common search actions
2023-08-19 18:37:15 +02:00
Earlopain
5f7f0309a4
[Tags] Remove fields that result in additional query from legacy parser
2023-08-19 15:59:38 +02:00
Earlopain
cfd7fb7941
[QueryBuilder] Use consistent structure for parsed fields
...
Makes it easier to generalize
2023-08-19 15:34:02 +02:00
Earlopain
d579c845f3
[Cleanup] Remove duplicate tag query scan method
2023-08-19 11:20:00 +02:00
Earlopain
c9a084e0a3
[Tags] Take category id instead of name for related search
...
Also have the bulk response return something with more structure to it
2023-08-18 22:40:06 +02:00
Earlopain
08bb60abc3
[Cleanup] Remove useless XML serializer includes
...
We can also remove to_xml from the `ApplicationDecorator` since there
is nothing that is or will be returning XML.
2023-08-18 19:50:34 +02:00
Earlopain
36eb87b084
[Cleanup] Move tag data outside of config file
...
Most of that is not easily configurable and requires db fields, css classes etc.
Everything in TagCategory was also pretty much incomprehensible for me
thanks to the indirection, so I moved it all inline instead.
I'll clean that up in later commits.
2023-08-18 19:08:52 +02:00
Earlopain
d6bfa02694
[Replacements] Fix approving when hitting the upload limit
...
Also improves the error message that was being returned because
just saying 'pending' isn't really helpful
2023-08-11 17:39:33 +02:00
Earlopain
7adac7b250
[Gems] Bump rubocop/regenerate todo
...
987 files inspected, 6509 offenses detected, 6224 offenses autocorrectable
2023-08-01 19:32:24 +02:00
Earlopain
42abd4ca33
[UserRevert] Fix error when user has uploaded a post
...
Extended version of #530
Co-authored-by: Donovan Daniels <hewwo@yiff.rocks>
2023-08-01 18:20:39 +02:00
Earlopain
7b36958ca5
[Discord] Fix gramar for janitor report
2023-07-26 17:25:42 +02:00
Earlopain
3b775e48bd
[Discord] Improve janitor report
...
* Make numbers bold
* Add count of pending flags/replacements
* Don't show negative numbers as diff
2023-06-27 15:55:53 +02:00
Donovan Daniels
55998374f7
[Discord] Add daily janitor reports ( #531 )
...
* Add daily janitor reports
* More specific config name, improve formatting
---------
Co-authored-by: Earlopain <earlopain@protonmail.com>
2023-06-26 21:10:28 +02:00
Earlopain
75c2adc2ae
[IQDB] Handle EHOSTUNREACH
2023-06-15 17:43:24 +02:00
Earlopain
e154c70e6c
Merge pull request #527 from faucetlol2/e
...
[Tags] Disallow é in tag names
2023-06-13 22:00:01 +02:00
Earlopain
d5b9b5fc2b
[Users] Remove unused/unnecessary roles
...
* Banned users can't login/is unused
* Voting made sense to be separated for danbooru, but not us
* Verified is unused
Previously these would also be part of the body attributes, but I can't
see how they would be useful there
2023-06-12 18:42:40 +02:00
faucet
cbe809fed5
[Tags] Disallow é in tag names
2023-06-10 17:46:04 +01:00
Earlopain
7ba84504b5
[Users] Better fix for setting ip on login for banned users
2023-06-08 13:11:54 +02:00
Earlopain
af1ee8382f
[Search] Rename exact_count parameter to total_count
...
Mimics the parameter in PaginatedArray and is now named same as the
method it's used in
2023-06-07 21:28:25 +02:00
Earlopain
86c28ef180
[Users] Don't update last ip on session create
...
This updates the ip of banned users which is undesirable.
The initialization in the user class interferes with tests and
is not actually needed. The controller already does this.
2023-06-06 20:12:02 +02:00
Earlopain
9035f845ee
[Search] Fix sequential prev/next buttons
...
Apparently this broke with the upgrade to rails 7, see cae6599631
2023-05-29 20:12:14 +02:00
Earlopain
85206a4023
[Search] Refactor pagination code
...
* Get rid of the individual extending for active_record
* Pass the pagination mode all the way down, especially in PaginatedArray
* Match PaginatedArray params to the base extensions
* Don't load total_count unless needed, mainly for api access
This should hopefully make it easier to understand
2023-05-29 18:36:46 +02:00
Earlopain
2f754df65c
[Search] Fix page=b0 returning results
2023-05-28 18:48:59 +02:00
Earlopain
b8f79a2acc
[Admin] Fix bogus permission check for promoting to admin
...
The error message is quite nice but unfortunatly the code didn't reflect that
2023-05-23 19:39:56 +02:00
Earlopain
b9ee500ec5
[Cleanup] Use same signature for cache methods as rails
2023-05-19 22:53:20 +02:00
