Docker(/Compose) can do mounts directly to the container as well.
https://blog.stefandroid.com/2021/03/03/mount-nfs-share-in-docker-compose.html
https://docs.docker.com/engine/storage/volumes/#create-a-service-which-creates-an-nfs-volume
I’ve used NFS mounts with docker compose before but I see the second link also includes an example for CIFS as well.
Named volumes are often the default because there is no chance of them conflicting with other services or containers running on the system.
Say you deployed two different docker compose apps each with their own MariaDB. With named volumes there is zero chance of those conflicting (at least from the filesystem perspective).
This also better facilitates easier cleanup. The apps documentation can say “docker compose down -v”, and they are done. Instead of listing a bunch of directories that need to be cleaned up.
Those lingering directories can also cause problems for users that might have wanted a clean start when their app is broken, but with a bind mount that broken database schema won’t have been deleted for them when they start up the services again.
All that said, I very much agree that when you go to deploy a docker service you should consider changing the named volumes to standard bind mounts for a couple of reasons.
When running production applications I don’t want the volumes to be able to be cleaned up so easily. A little extra protection from accidental deletion is handy.
The default location for named volumes doesn’t work well with any advanced partitioning strategies. i.e. if you want your database volume on a different partition than your static web content.
Old reason and maybe more user preference at this point but back before the docker overlay2 storage driver had matured we used the btrfs driver instead and occasionally Docker would break and we would need to wipe out the entire /var/lib/docker btrfs filesystem, so I just personally want to keep anything persistent out of that directory.
So basically application writers should use named volumes to simplify the documentation/installation/maintenance/cleanup of their applications.
Systems administrators running those applications should know and understand the docker compose well enough to change those settings to make them production ready for their environment. Reading through it and making those changes ends up being part of learning how the containers are structured in the first place.
For shared lines like cable and wireless it is often asymmetrical so that everyone gets better speeds, not so they can hold you back.
For wireless service providers for instance let’s say you have 20 customers on a single access point. Like a walkie-talkie you can’t both transmit and receive at the same time, and no two customers can be transmitting at the same time either.
So to get around this problem TDMA (time division multiple access) is used. Basically time is split into slices and each user is given a certain percentage of those slices.
Since the AP is transmitting to everyone it usually gets the bulk of the slices like 60+%. This is the shared download speed for everyone in the network.
Most users don’t really upload much so giving the user radios equal slices to the AP would be a massive waste of air time, and since there are 20 customers on this theoretical AP every 1mbit cut off of each users upload speed is 20mbit added to the total download capability for anyone downloading on that AP.
So let’s say we have APs/clients capable of 1000mbit. With 20 users and 1AP if we wanted symmetrical speeds we need 40 equal slots, 20 slots on the AP one for each user to download and 1 slot for each user to upload back. Every user gets 25mbit download and 25mbit upload.
Contrast that to asymmetrical. Let’s say we do a 80/20 AP/client airtime split. We end up with 800mbit shared download amongst everyone and 10mbit upload per user.
In the worst case scenario every user is downloading at the same time meaning you get about 40mbit of that 800, still quite the improvement over 25mbit and if some of those people aren’t home or aren’t active at the time that means that much more for those who are active.
I think the size of the slices is a little more dynamic on more modern systems where AP adjusts the user radios slices on the fly so that idle clients don’t have a bunch of dead air but they still need to have a little time allocated to them for when data does start to flow.
A quick Google seems to show that DOCSIS cable modems use TDMA as well so this all likely applies to cable users as well.
The point is that you can still treat it like a physical game. So there are upsides in that you can borrow it to your friends or resell it.
If it is a game that gets updated often or requires updates to even play it (multiplayer games) then having the game data on the card is next to worthless anyways and just makes publishing the game more difficult because they can’t start manufacturing the cards until the game is 100% ready.
Nintendo’s audience goes for physical much more than the other consoles, much easier swapping cards than dealing with family sharing, a lot of their adult users collect games, and generally Nintendo games hold their value much more so being able to resell is important. So this is a compromise between what their users want and what they need for modern game development.
Slippery slope for sure if they start doing the same with single player games but there are valid reasons for them to do this, and the alternative is they just start forcing everyone to download all of their games which is even worse. MIG switch would never have been an issue for them if there just weren’t game card slots to begin with.
Of course end users should assume the store is going to get shutdown someday and their games will be inaccessible at that time. Nintendo needs to shutdown those stores so that a couple of generations later they can sell everyone the same games for the second/third/fourth time.
Depending on how you setup your reverse proxy it can reduce random scanning/login attempts to basically zero. The point of a reverse proxy is to act as a proxy, as a sort of web router, and to validate that the http requests are correctly formatted.
For the routing depending on what DNS name/path the request comes in with it can route to different backends. So you can say that app1.yourdomain.com is routed to the internal IP address of your app1, and app2.yourdomain.com goes to app2. You can also do this with paths if the applications can handle it. Like yourdomain.com/app1.
When your client makes a request the reverse proxy uses the “Host” header or the SNI string that is part of the TLS connection to determine what certificate to use and what application to route to.
There is usually a “default” backend for any request that doesn’t match any of the names for your backend services (like a scanner blindly trying to access your IP). If you disable the default backend or redirect default requests to something that you know is secure any attacker scanning your IP for vulnerabilities would get their requests rejected. The only way they can even try to hit your service is to know the correct DNS name of your service.
Some reverse proxies (Traefik, HAproxy) have options to reject the requests before the TLS negation has even completed. If the SNI string doesn’t match the connection just drops it doesn’t even bother to send a 404/5xx error. This can prevent an attacker from doing information gathering about the reverse proxy itself that might be helpful in attacking it.
This is security by obscurity which isn’t really security, but it does reduce your risk because it significantly reduces the chances of an attacker being able to find your applications.
Reverse proxies also have a much narrower scope than most applications as well. Your services are running a web server with your application, but is Jellyfin’s built in webserver secure? Could an attacker send invalid data in headers/requests to trigger a buffer overflow? A reverse proxy often does a much better job of preventing those kinds of attacks, rejecting invalid requests before they ever get to your application.
Agreed. The nonstandard port helps too. Most script kiddies aren’t going to know your service even exists.
Take it another step further and remove the default backend on your reverse proxy so that requests to anything but the correct DNS name are dropped (bots just are probing IPs) and you basically don’t have to worry at all. Just make sure to keep your reverse proxy up to date.
The reverse proxy ends up enabling security through obscurity, which shouldn’t be your only line of defence, but it is an effective first line of defence especially for anyone who isn’t a target of foreign government level of attacks.
Adding basic auth to your reverse proxy endpoints extends that a whole lot further. Form based logins on your apps might be a lot prettier, but it’s a lot harder to probe for what’s running behind your proxy when every single URI just returns 401. I trust my reverse proxy doing basic auth a lot more than I trust some php login form.
I always see posters on Lemmy about setting up elaborate VPN setups for as the only way to access internal services, but it seems like awful overkill to me.
VPN still needed for some things that are inherently insecure or just should never be exposed to the outside, but if it is a web service with authentication required a reverse proxy is plenty of security for a home lab.
You are paying for reasonably well polished software, which for non technical people makes them a very good choice.
They have one click module installs for a lot of the things that self hosted people would want to run. If you want Plex, a onedrive clone, photo sync on your phone, etc just click a button and they handle installing and most of the maintenance of running that software for you. Obviously these are available on other open source NAS appliances now too so this isn’t much of a differnentiator for them anymore, but they were one of the first to do this.
I use them for their NVR which there are open source alternatives for but they aren’t nearly as polished, user friendly, or feature rich.
Their backup solution is also reasonably good for some home labs and small business use cases. If you have a VMware lab at home for instance it can connect to your vCenter and it do incremental backups of your VMs. There is an agent for Windows machines as well so you can keep laptops/desktops backed up.
For businesses there are backup options for Office365/Google Workspace where it can keep backups of your email/calendar/onedrive/SharePoint/etc. So there are a lot of capabilities there that aren’t really well covered with open source tools right now.
I run my own built NAS for mass storage because anything over two drives is way too expensive from Synology and I specifically wanted ZFS, but the two drive units were priced low enough to buy just for the software. If you want a set and forget NAS they were a pretty good solution.
If their drives are reasonably priced maybe they will still be an okay choice for some people, but we all know the point of this is for them to make more money so that is unlikely. There are alternatives like Qnap, but unless you specifically need one of their software components either build it yourself or grab one of the open source NAS distros.
If your NAS has enough resources the happy(ish) medium is to use your NAS as a hypervisor. The NAS can be on the bare hardware or its own VM, and the containers can have their own VMs as needed.
Then you don’t have to take down your NAS when you need to reboot your container’s VMs, and you get a little extra security separation between any externally facing services and any potentially sensitive data on the NAS.
Lots of performance trade offs there, but I tend to want to keep my NAS on more stable OS versions, and then the other workloads can be more bleeding edge/experimental as needed. It is a good mix if you have the resources, and having a hypervisor to test VMs is always useful.
If you are just using a self signed server certificate anyone can connect to your services. Many browsers/applications will fail to connect or give a warning but it can be easily bypassed.
Unless you are talking about mutual TLS authentication (aka mTLS or two way ssl). With mutual TLS in addition to the server key+cert you also have a client key+cert for your client. And you setup your web server/reverse proxy to only allow connections from clients that can prove they have that client key.
So in the context of this thread mTLS is a great way to protect your externally exposed services. Mutual TLS should be just as strong of a protection as a VPN, and in fact many VPNs use mutual TLS to authenticate clients (i.e. if you have an OpenVPN file with certs in it instead of a pre-shared key). So they are doing the exact same thing. Why not skip all of the extra VPN steps and setup mTLS directly to your services.
mTLS prevents any web requests from getting through before the client has authenticated, but it can be a little complicated to setup. In reality basic auth at the reverse proxy and a sufficiently strong password is just as good, and is much easier to setup/use.
Here are a couple of relevant links for nginx. Traefik and many other reverse proxies can do the same.
The biggest question is, are you looking for Dolby Vision support?
There is no open source implementation for Dolby Vision or HDR10+ so if you want to use those formats you are limited to Android/Apple/Amazon streaming boxes.
If you want to avoid the ads from those devices apart from side loading apks to replace home screens or something the only way to get Dolby Vision with Kodi/standard Linux is to buy a CoreELEC supported streaming device and flashing it with CoreELEC.
List of supported devices here
CoreELEC is Kodi based so it limits your player choice, but there are plugins for Plex/Jellyfin if you want to pull from those as back ends.
Personally it is a lot easier to just grab the latest gen Onn 4k Pro from Walmart for $50 and deal with the Google TV ads (never leave my streaming app anyways). Only downside with the Onn is lack of Dolby TrueHD/DTS Master audio output, but it handles AV1, and more Dolby Vision profiles than the Shield does at a much cheaper price. It also handles HDR10+ which the Shield doesn’t but that for at isn’t nearly as common and many of the big TV brands don’t support it anyways.
Like most have said it is best to stay away from ZFS deduplication. Especially if your data set is media the chances of an entire ZFS block being the same as any other is small unless you somehow have multiple copies of the same content.
Imagine two mp3s with the exact same music content but with slightly different artist metadata. A single bit longer or shorter at the beginning of the file and even if the file spans multiple blocks ZFS won’t be able to duplicate a single byte. A single bit offsetting the rest of the file just a little is enough to throw off the block checksums across every block in the file.
To contrast with ZFS, enterprise backup/NAS appliances with deduplication usually do a lot more than block level checks. They usually check for data with sliding window sizes/offsets to find more duplicate data.
There are still some use cases where ZFS can help. Like if you were doing multiple full backups of VMs. A VM image has a fixed size so the offset issue above isn’t an issue, but if beware that enabling deduplication for even a single ZFS filesystem affects the entire pool, even ZFS filesystems that have deduplication disabed. The deduplication table is global for the pool and once you have turned it on you really can’t get rid of it. If you get into a situation where you don’t have enough memory to keep the deduplication table in memory ZFS will grind to a halt and the only way to completely remove deduplication is to copy all of your data to a new ZFS pool.
If you think this feature would still be useful for you, you might want to wait for 2.3 to release (which isn’t too far off) for the new fast dedup feature which fixes or at least prevents a lot of the major issues with ZFS dedup
More info on the fast dedup feature here https://github.com/openzfs/zfs/discussions/15896
Slows down then freezes sure sounds like an out of memory situation, so to add to yours here they might actually want less swap. Sometimes you would rather hit the oom killer sooner instead of waiting on swap to fill.
Ideally login via SSH from another machine to figure out what is using the memory (hopefully the system is responsive enough for SSH), and if it is your critical programs causing the problem then you should consider a memory upgrade.
Contrary to a lot of posts that I have seen, I would say ZFS isn’t pointless with a single drive. Even if you can’t repair corruption with a single drive knowing something is corrupt in the first place is even more important (you have backups to restore it from right?).
And a ZFS still has a lot of features that are useful regardless. Like snapshots, compression, reflinks, send/receive, and COW means no concerns about data loss during a crash.
BTRFS can do all of this too and I believe it is better about low memory systems but since you have ZFS on your NAS you unlock a lot of possibilities keeping them the same.
I.e. say you keep your T110ii running with ZFS you can use tools like syncoid to periodically push snapshots from the Optiplex to your T110.
That way your Optiplex can be a workhorse, and your NAS can keep the backup+periodic snapshots of the important data.
I don’t have any experience with TrueNAS in particular but it looks like syncoid works with it. You might need to make sure that pool versions/flags are the same for sending/receive to work.
Alternatively keep that data on an NFS mount. The SSD in the Optiplex would just be for the base OS and wouldn’t have any data that can’t be thrown away. The disadvantage here being your Optiplex now relies on a lot more to keep running (networking + nas must be online all the time).
If you need HA for the VMs you likely need distributed storage for the VMs to run on. No point in building an HA VM solution if it just moves the single point of failure to your NAS.
Personally I like Harvester, but the minimum requirements are probably beyond what your hardware can handle.
Since you are already on TrueNAS Scale have you looked at using TrueNAS Scale on the Optiplex with replication tasks for backups?
If you are accessing your files through dolphin on your Linux device this change has no effect on you. In that case Synology is just sharing files and it doesn’t know or care what kind of files they are.
This change is mostly for people who were using the Synology videos app to stream videos. I assume Plex is much more common on Synology and I don’t believe anything changed with Plex’s h265 support.
If you were using the built in Synology videos app and have objections to Plex give Jellyfin a try. It should handle h265 and doesn’t require a purchase like Plex does to unlock features like mobile apps.
Linux isn’t dropping any codecs and should be able to handle almost any media you throw at it. Codec support depends on what app you are using, and most Linux apps use ffmpeg to do that decoding. As far as I know Debian hasn’t dropped support for h265, but even if they did you could always compile your own ffmpeg libraries with it re-enabled.
How can I most easily search my NAS for files needing the removed codecs
The mediainfo command is one of the easiest ways to do this on the command line. It can tell you what video/audio codecs are used in a file.
With Linux and Synology DSM both dropping codecs, I am considering just taking the storage hit to convert to h.264 or another format. What would you recommend?
To answer this you need to know the least common denominator of supported codecs on everything you want to play back on. If you are only worried about playing this back on your Linux machine with your 1080s then you fully support h265 already and you should not convert anything. Any conversion between codecs is lossy so it is best to leave them as they are or else you will lose quality.
If you have other hardware that can’t support h265, h264 is probably the next best. Almost any hardware in the last 15 years should easily handle h264.
When it comes to thumbnails for a remote filesystem like this are they generated and stored on my PC or will the PC save them to the folder on the NAS where other programs could use them.
Yes they are generated locally, and Dolphin stores them in ~/.cache/thumbnails on your local system.
In any KDE app you can connect with SFTP in the open file dialog. Just type sftp://user@server/path and you can browse/open/edit files the remote server. ssh keys+agent make things a lot easier here obviously.
I believe so. The package descriptions for most of the ZFS packages in Ubuntu mention OpenZFS, so it certainly appears that way.
You can still create pools that are compatible with Oracle Solaris, you just have to set the pool version to 28 or older when you create it and obviously don’t update it. That will prevent you from using any of the newer features that have been added since the fork.
Well worse than that, Oracle closed sourced ZFS, so OpenZFS was forced to become a fork, and they are no longer compatible with each other.
As for GPL the CDDL license that ZFS uses made sure that code contributions attribute copyright to the project owners, which means they can change the license as they please without having to track down contributors.
You would think with their investments in Oracle Linux and btrfs they would welcome that license change, but apparently they need excuses to keep putting money into Solaris, and their Oracle ZFS appliances instead.
Unauthorized VPNs (non government approved) are illegal in China. If a business needs their own they can get approval but they have to apply for those exceptions.
It isn’t really enforced, probably especially so for non citizens, but if you do something they don’t like it is something they could use against you.
You would probably be less breaking the law to just directly open up SSH and access that instead of tunneling through a VPN. Even though SSH can do tunneling of its own.