I thought it would be great to have a comparison between _JAVA_OPTIONS and JAVA_TOOL_OPTIONS.
I have been searching a bit for one, but I cannot find anything, so I hope we can find the knowledge here on Stackoverflow.

JAVA_OPTS is included for completeness. It is not part of the JVM, but there is a lot of questions about it out in the wild.

What I know:

So far I have found out that:

• JAVA_OPTS is not used by the JDK, but by a bunch of other apps (see this post).
• JAVA_TOOL_OPTIONS and _JAVA_OPTIONS are ways to specify JVM arguments as an environment variable instead of command line parameters.
  • The are picked up by at least java and javac
  • They have this precedence:
    1. _JAVA_OPTIONS (overwrites the others)
    2. Command line parameters
    3. JAVA_TOOL_OPTIONS (is overwritten by the others)

What I would like to know

• Are there any official documentation comparing JAVA_TOOL_OPTIONS and _JAVA_OPTIONS
• Are there any other differences between JAVA_TOOL_OPTIONS and _JAVA_OPTIONS (except from precedence).
• Which executables pick up JAVA_TOOL_OPTIONS and _JAVA_OPTIONS (in addition to java and javac)
• Any limitation on what can be included on JAVA_TOOL_OPTIONS and _JAVA_OPTIONS

Official Documentation

I have not been able to find any documentation about _JAVA_OPTIONS. The documentation for JAVA_TOOL_OPTIONS does not shed much light on the difference:

Since the command-line cannot always be accessed or modified, for example in embedded VMs or simply VMs launched deep within scripts, a JAVA_TOOL_OPTIONS variable is provided so that agents may be launched in these cases.
…

Example script

This is the code I used to figure this out. Console output is included as comments:

export JAVA_OPTS=foobar
export JAVA_TOOL_OPTIONS= 
export _JAVA_OPTIONS="-Xmx512m -Xms64m"

java -version                          
# Picked up JAVA_TOOL_OPTIONS: 
# Picked up _JAVA_OPTIONS: -Xmx512m -Xms64m
# java version "1.7.0_40"
OpenJDK Runtime Environment (IcedTea 2.4.1) (suse-3.41.1-x86_64)
OpenJDK 64-Bit Server VM (build 24.0-b50, mixed mode)

javac -version
# Picked up JAVA_TOOL_OPTIONS: 
# Picked up _JAVA_OPTIONS: -Xmx512m -Xms64m
# javac 1.7.0_40

export JAVA_TOOL_OPTIONS="-Xmx1 -Xms1"
export _JAVA_OPTIONS="-Xmx512m -Xms64m"
javac -version
# Picked up JAVA_TOOL_OPTIONS: -Xmx1 -Xms1
# Picked up _JAVA_OPTIONS: -Xmx512m -Xms64m
# javac 1.7.0_40

export JAVA_TOOL_OPTIONS="-Xmx512m -Xms64m"
export _JAVA_OPTIONS="-Xmx1 -Xms1"
javac -version
# Picked up JAVA_TOOL_OPTIONS: -Xmx512m -Xms64m
# Picked up _JAVA_OPTIONS: -Xmx1 -Xms1
# Error occurred during initialization of VM
# Too small initial heap

export JAVA_TOOL_OPTIONS="-Xmx1 -Xms1"
export _JAVA_OPTIONS=
java -Xmx512m -Xms64m -version
# Picked up JAVA_TOOL_OPTIONS: -Xmx1 -Xms1
# Picked up _JAVA_OPTIONS: 
# java version "1.7.0_40"
# OpenJDK Runtime Environment (IcedTea 2.4.1) (suse-3.41.1-x86_64)
# OpenJDK 64-Bit Server VM (build 24.0-b50, mixed mode)

export JAVA_TOOL_OPTIONS=
export _JAVA_OPTIONS="-Xmx1 -Xms1"
java -Xmx512m -Xms64m -version
# Picked up JAVA_TOOL_OPTIONS: 
# Picked up _JAVA_OPTIONS: -Xmx1 -Xms1
# Error occurred during initialization of VM
# Too small initial heap

You have pretty much nailed it except that these options are picked up even if you start JVM in-process via a library call.

The fact that _JAVA_OPTIONS is not documented suggests that it is not recommended to use this variable, and I’ve actually seen people abuse it by setting it in their ~/.bashrc. However, if you want to get to the bottom of this problem, you can check the source of Oracle HotSpot VM (e.g. in OpenJDK7).

You should also remember that there is no guarantee other VMs have or will continue to have support for undocumented variables.

UPDATE 2015-08-04: To save five minutes for folks coming from search engines, _JAVA_OPTIONS trumps command-line arguments, which in turn trump JAVA_TOOL_OPTIONS.

There is one more difference: _JAVA_OPTIONS is Oracle specific. IBM JVM is using IBM_JAVA_OPTIONS instead. This was probably done to be able to define machine-specific options without collisions. JAVA_TOOL_OPTIONS is recognized by all VMs.

JAVA_OPTS have no special handling in JVM at all.

And according to https://bugs.openjdk.java.net/browse/JDK-4971166 the JAVA_TOOL_OPTIONS is included in standard JVMTI specification, does better handling of quoted spaces and should be always preferred instead of undocumented Hotspot-specific _JAVA_OPTIONS.

Also beware that using these prints additional message to stdout that can’t be suppressed.

As @ryenus noted, since JDK 9+, there’s JDK_JAVA_OPTIONS as the preferred replacement, see What is the difference between JDK_JAVA_OPTIONS and JAVA_TOOL_OPTIONS when using Java 11?