edshot99
01d9e47d0a
Merge branch 'master' into master2
2025-01-16 19:56:41 -06:00
Donovan Daniels
b44f8a7d6f
[Posts] Add contributor category ( #794 )
2024-12-18 06:07:30 -08:00
edshot99
4928747b83
add transcript support
...
it could be possible to load closed captions and subtitles from the transcript div via javascript, but I hate javascript so perhaps some other time.
2024-11-10 13:58:15 -06:00
edshot99
b7fee5fe2f
add title support
2024-10-30 18:52:53 -05:00
edshot99
a3e00b591f
add new tag category 'People'
2024-10-28 20:04:58 -05:00
Donovan Daniels
f315504660
[Posts] Add verified artist search ( #663 )
2024-09-07 13:34:25 -07:00
Earlopain
6dfd8d8a7e
[Rails] Use different syntax to query if pg array contains single value
...
This will be broken in the next major rails version: https://github.com/rails/rails/issues/51617
2024-04-21 18:21:37 +02:00
Earlopain
584f69d407
[Posts] Fix initial import including deleted notes
2024-04-05 13:18:45 +02:00
Earlopain
b6dc473a55
[Elasticsearch] Bring back the proxy object
2023-09-16 17:09:33 +02:00
Earlopain
0bd651c08b
[Elasticsearch] Remove elasticsearch-model gem
...
This should be the bulk of the work to integrate with opensearch-ruby
2023-09-16 15:52:05 +02:00
Earlopain
ddcb7c1af1
[Elasticsearch] Implement index_name
2023-09-16 15:39:49 +02:00
Earlopain
f1d5864724
[Elasticsearch] Move more code out of indexable
2023-09-16 14:26:04 +02:00
Earlopain
5d791f9803
[Elasticsearch] Extend the base class
...
This whole thing needs some refactoring after I'm done with this.
2023-09-16 14:16:08 +02:00
Earlopain
bd1bd8cfc3
[Elasticsearch] Stop sending the document type
...
This is deprecated in ES7 and removed in ES8 and OS2
2023-09-16 13:48:30 +02:00
Earlopain
a8b085e714
[Elasticsearch] Implement delete_document
2023-09-16 13:23:25 +02:00
Earlopain
123a96bd26
[Elasticsearch] Implement index_document
...
Also makes the callbacks for update the same as for create.
I don't see why creation should run inline, same for reownering.
2023-09-16 13:19:21 +02:00
Earlopain
3285d546f9
[Elasticsearch] Move index creation to DocumentStore
2023-09-14 21:53:26 +02:00
Earlopain
a79e25dfbd
[Elasticsearch] Start implementing own elasticsearch-model
...
This implements create_index!
2023-09-14 20:49:19 +02:00
Earlopain
5201008347
[Elastic] Get rid of the prod special-case for index names
...
Aliases have been created to handle this.
Run this on your own instance to create the missing indicies
```
Post.__elasticsearch__.create_index!
PostVersion.__elasticsearch__.create_index!
Post.import
PostVersion.import
```
2023-05-30 18:43:45 +02:00
Earlopain
31b27ac1fd
[Posts] Remove pixiv_id from elastic index
...
There's no trouble with just removing it in code. Eventually switching
OpenSearch will clean this up properly
2023-05-30 16:13:44 +02:00
Earlopain
18382d710e
[Sidekiq] Remove duplicate index update job
...
The only real difference was the queue, that can be set dynamically. Also
moves the actual work into it's own method to make execution flow more obvious
2023-05-02 21:31:07 +02:00
Earlopain
0c739a90ae
[Sidekiq] Switch back to enqueuing the proper class
2023-05-01 18:19:26 +02:00
Earlopain
c4e33651c6
[Sidekiq] Enqueue the previously added jobs
2023-05-01 16:42:21 +02:00
Zwagoth
5187d4b15c
Revert "[Sidekiq] First step to convert jobs to ActiveJob form"
2023-01-17 03:18:36 -05:00
Earlopain
bf8fa55094
[Sidekiq] First step to convert jobs to ActiveJob form
...
See https://keygen.sh/blog/how-to-change-the-signature-of-a-sidekiq-job
2023-01-13 19:57:20 +01:00
Earlopain
b0be3b1946
[Cleanup] Remove unnecessary requires
2022-12-17 17:42:09 +01:00
Earlopain
4cb88300c7
[Versions] Rename post/pool archive classes to versions
...
This just makes sense, and was confusing beforehand.
Case-in-point: Two test classes for pools: pool_versions and pool_archive
2022-08-06 18:58:24 +02:00
Earlopain
c3224e111e
Revert "[PostVersions] Fix result window on search"
...
This reverts commit d53a0ead98
2022-04-21 21:41:24 +02:00
Earlopain
d53a0ead98
[PostVersions] Fix result window on search
...
Also see afbe5bb7e9
2022-04-13 12:26:11 +02:00
Earlopain
841610cd90
[Tests] Fix a few tests related to elasticsearch
...
Namespace the indicies, this should stop the search being messed up
after running tests. Also refresh the index immediately, this slows down
tests a bit but gets rid of a few failures. This whole search testing needs
to be revised some time.
Run this to fix your development setup:
```
Post.__elasticsearch__.create_index!
Post.import
PostArchive.__elasticsearch__.create_index!
PostArchive.import
```
2022-04-05 19:24:31 +02:00
Earlopain
aa8e1037bf
[Cleanup] Remove pixiv_id
2022-03-16 17:11:10 +01:00
Earlopain
147ec019e8
[Search] Add pending replacements field
2021-10-05 21:14:00 +02:00
Kira
dbb5870d93
[Tests] Fix pool related errors in tests
2021-02-25 12:08:09 -08:00
Kira
58e8fe22fd
Tests and fixes
2021-02-25 12:08:08 -08:00
Kira
015b81203a
[Posts] Support video duration
2020-10-04 12:29:22 -07:00
Kira
a6b21361b5
[Indexer] lowecase deletion reasons
2020-05-08 18:44:02 -07:00
Kira
370018b25d
Script to generate missing post version fields
2020-03-01 15:43:35 -08:00
Kira
8e6840a54f
Fix import error with invalid height/width
2020-02-01 10:48:44 -08:00
Kira
ac5361f169
Make it so set and pool import can run in real time
2020-02-01 10:17:14 -08:00
Kira
c6b1046fc5
Add a few boolean search tags
2020-01-25 03:53:53 -08:00
Kira
8e82f53ca2
Fix bad index on post flags
2019-10-21 03:39:55 -07:00
Kira
81e7d10aa6
Add reason and description to index
2019-10-09 11:12:09 -07:00
Kira
f5f6973011
Add search to post versions
2019-10-09 08:34:22 -07:00
Kira
6bfe7e12e9
Fix post version indexer options
...
Up batch size to 10k to vastly improve indexing speed.
Fix reversed from and to options.
2019-10-09 07:51:56 -07:00
Kira
afbe5bb7e9
Fix result window on search
...
The default result window is 10k in ES, which is below our max
result count of 320 posts over 750 pages. Added 10k to the cap
just in case something comes up and we need more results.
I have been told that this doesn't cause problems in production
on Derpi, but we'll see. This can be lowered at runtime, so it
won't be a big deal if we have to roll it back down again.
2019-09-25 21:04:50 -07:00
Kira
f85de16f96
Add missing search options
...
Add note, deletedby, and delreason search options.
2019-09-25 20:40:52 -07:00
Kira
5a133e1643
Add lore tag category
2019-09-24 17:04:46 -07:00
Kira
46bacc13d8
Fix ES import for pools/sets
...
The SQL operator used in the import script was trying to ensure
that all elements were contained within each pool/set array, but
the script was actually interested in any intersection between the
two arrays.
2019-09-15 13:15:38 -07:00
Kira
4da805a4fa
Use elasticsearch for post history
2019-09-13 19:50:26 -07:00
Kira
a8503bee23
Fix elasticsearch for ES 7.x
2019-09-09 12:57:57 -07:00
