Not able to push

[noszczykmichal]

Hi,

about a year ago I was writing with you about not working instance of this api. The same thing happened recentely so I decided to lunch my own instance to not bother you any more. However when working with your repo and specificly with script "rdf_parser.py" I've noticed that it doesn't work good when name of the file can't be converted to the integer. The thing is that when I downloaded fresh PG catalogue I've found out that in main catalogue apart from directories with rdf files there is one named 'test'. On reaching it your script crashed and as it seems ocassionally can ommit some rdf-files. I've prepered some fix on seperate branch but got error about lack of permissions to push so you can either add me to contributors (I guess then I will be able to push my changes) or you can apply them directly if you think they work for you. Changes start from line 133:
"if filename.endswith(".rdf"):
try:
id = int(os.path.basename(path))
input_file = os.path.join(path, filename)
output_file = os.path.join(args.output, f"{id}.json")
Book.convert_to_json(id, input_file, output_file)
print(f"Converted book: {id}")
except ValueError:
print(f"Error: Cannot convert file named '{os.path.basename(path)}' to integer")
continue

if name == 'main':
parser = argparse.ArgumentParser(
description="Convert RDF files to JSON format for faster parsing.")
parser.add_argument("-i",
"--input", help="The input RDF directory.",
default="res/rdf")
parser.add_argument(
"-o", "--output", help="The output JSON directory", default="res/json")
main(parser.parse_args())" I'm not an python expert so pardon me if I mixed up something.

[GnikDroy]

Thanks for the heads up. I think I renewed the lease a couple of weeks back, but if you want 100% reliability, you are indeed better off creating your own instance.

I understand the issue. The usual way to contribute to is through a pull request. So, if you want to send in your patch you can just create a pull request of your branch.

I will also try to release a new sqlite3 database, so people will get a more upto date version of the catalogue.

[noszczykmichal]

Yeah, you were right - sorry, I'm quite new to contributing to someone else's repo. After creating a fork and then creating a pull request, everything worked as it should. And yeah, you're right that now the API works - it started working when I was in the middle of my work on my own instance. Nevertheless, I've decided to proceed anyway as I wouldn't like to rely on your instance. Apart from that, even if I use it from time to time, I wouldn't like to use your quota for the server or whatever way of charging users they use. Anyway, I will write to you in a few days (currently I'm a bit busy) because I have some issues while trying to load the generated DB, and I'm not sure where the problem originates from. I guess I'm doing something wrong, but because I barely know Python and databases, I'm not aware of the fact.

[noszczykmichal]

Ok, so as I said I have an issue while trying to load generated db. This is command I'm trying to execute: "python manage.py load_db --clear C:\Users\mike8\Downloads\data_base\booksDataBase.db". And the script fails with error: "django.db.utils.IntegrityError: UNIQUE constraint failed: api_agenttype.name"
In attachement the output I get from terminal while trying to execute load_db script:


Since you've successfully created yours instance I don't expect it is somekind of the issue with your code, but maybe they (the PG team) changed something in the data they provide that now causes the issue. I would greatly appreciate your assistance here or any other suggestions on what may cause the issue.

[GnikDroy]

Apologies for replying late. I will take a look at this today. Will also release a more recent snapshot DB.