I want to have a mirror of my local music collection on my server, and a script that periodically updates the server to, well, mirror my local collection.

But crucially, I want to convert all lossless files to lossy, preferably before uploading them.

That’s the one reason why I can’t just use git - or so I believe.

I also want locally deleted files to be deleted on the server.

Sometimes I even move files around (I believe in directory structure) and again, git deals with this perfectly. If it weren’t for the lossless-to-lossy caveat.

It would be perfect if my script could recognize that just like git does, instead of deleting and reuploading the same file to a different location.

My head is spinning round and round and before I continue messing around with find and scp it’s time to ask the community.

I am writing in bash but if some python module could help with it I’m sure I could find my way around it.

TIA

additional info:

  • Not all files in the local collection are lossless. A variety of formats.
  • The purpose of the remote is for listening/streaming with various applications
  • The lossy version is for both reducing upload and download (streaming) bandwidth. On mobile broadband FLAC tends to buffer a lot.
  • The home of the collection (and its origin) is my local machine.
  • The local machine cannot act as a server
  • DecentM@lemmy.ml
    8·
    1 month ago

    Don’t know of a solution that does this, but you could solve it with a two-step process. First, rsync the files to the server as-is, then use a background job on the server that converts lossless to lossy every hour or so.

    Storage is really cheap these days though, why compress lossy in the first place?

    • N0x0n@lemmy.ml
      1·
      1 month ago

      You can compress to lossy and still don’t make the difference between each while saving a ton of storage. Opus 192k is really good and mostly transparent.

      I do agree that storage is cheap however if you have to make backups, it really gets satured very fast !

  • Eager Eagle@lemmy.worldEnglish
    5·
    1 month ago

    I want to convert all lossless files to lossy, preferably before uploading them

    so it’s not exactly a mirror, right?

    here’s an idea:

    • A - the source containing lossless files
    • B - the local storage of lossy files
    • C - a remote mirror of B

    With that, you can do:

    • A -> B: a systemd service that makes this conversion.
    • B -> C: git or syncthing to mirror and/or version control.

    This uses more storage than you probably intended to (lossy files are also stored locally), but it’s a mirror.

    • A_norny_mousse@feddit.orgOP
      2·
      1 month ago

      so it’s not exactly a mirror, right?

      Correct!

      Your workflow is actually something I thought of but the duplication of all lossy files would be a bit too much and replacing them with symlinks would not work with git afaik.