Repository navigation
Possible supporting gpn series model? #1
Description
Hi, thanks for your great work. It seems that gpn model is not supported (gpn-msa, its new version supports human genome). Would you please consider including it? Thanks.
Traceback (most recent call last):
File "/gpfs/radev/project/ying_rex/tl688/BEND/run_testgpn.py", line 10, in <module>
embedder = bend.embedders.GPNEmbedder('songlab/gpn-msa-sapiens')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/radev/project/ying_rex/tl688/BEND/bend/utils/embedders.py", line 63, in __init__
self.load_model(*args, **kwargs)
File "/gpfs/radev/project/ying_rex/tl688/BEND/bend/utils/embedders.py", line 128, in load_model
self.tokenizer = AutoTokenizer.from_pretrained(model_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/radev/project/ying_rex/tl688/llm/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 913, in from_pretrained
tokenizer_class_py, tokenizer_class_fast = TOKENIZER_MAPPING[type(config)]
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/gpfs/radev/project/ying_rex/tl688/llm/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 732, in __getitem__
model_type = self._reverse_config_mapping[key.__name__]
KeyError: 'GPNRoFormerConfig'
This is the error message.
Activity
miguelgondu commentedon Sep 22, 2024
Tagging the relevant people: @frederikkemarin @fteufel
fteufel commentedon Sep 22, 2024
Hi,
yes, GPN-MSA is not supported. As indicated in the readme (https://github.com/MachineLearningLifeScience/BEND?tab=readme-ov-file#embedders-overview), the embedder is meant to be used with the A. thaliana/Brassicales models.
My last status was that it's unclear whether GPN-MSA is useful as an embedding model (https://x.com/gsbenegas/status/1727746984055083075). Supporting it in our embedders would be a bit more complicated, as it operates on MSAs, rather than single sequences.
fteufel commentedon Sep 22, 2024
@frederikkemarin can you replace the repo here with a fork from the main repo in your account, so that it points back correctly and can be synced?
HelloWorldLTY commentedon Sep 22, 2024
Hi thanks a lot, that makes sense to me. I notice that there is one paper custome trained a GPN for human genome, and I tried that model with BEND which worked for me. Thanks a lot.
https://www.biorxiv.org/content/10.1101/2024.02.29.582810v1.full.pdf