One of my most frequently-visited posts is "Retrieving Oracle patches with wget." Recently, a commenter on that post asked about using wget to download software from Oracle Technet (OTN). It's a little complicated to use wget to download from OTN, because there are cookies involved. This post discusses how to extract the relevant cookies from Firefox and Google Chrome, and use those cookies with wget to retrieve files. If you manage to wade through my excessive exposition, there's even a script at the end that you might find useful.

First things first: This is not a new idea. Pythian's Marc Fielding wrote an excellent article a long time ago about using wget and the text-based browser lynx to download files from OTN. His article is the reason that I even knew this was possible; I'm just expanding on the concept to include other browsers. If you have lynx installed on your server, you can stop reading this post, go read Marc's post instead, and you're all set. In fact, it's a good idea to read Marc's article anyway, because I'm going to reference it a few times in this post.

Overview and assumptions

As you've read from Marc's article (you did go to read that, right?), the basic process to set up downloads from OTN with wget is as follows:

1. Authenticate to OTN with your username and password. You need to do this for cookies to be set properly. No workarounds or backdoors here, you have to sign in.
2. Navigate to the relevant page in the Downloads section for the software to be downloaded
3. Click the radio button to accept the license agreement. This is necessary to reveal the download links so you can copy the URL to feed to wget. Again, no backdoors, no workarounds.
4. Copy the URLs of the files you want to download
5. Extract the cookies created during your browser session into a text file.
6. Invoke wget with the --load-cookies option to download the desired files.

By now, you might be thinking, "If it's that easy, why is the scroll bar next to this post so long?" The answer is that step 5 is somewhat involved.

I'm going to assume a few things. First, we'll be working with either Firefox or Google Chrome, because they store their cookies in similar ways. Safari uses XML to store cookies, and Internet Explorer is just an abomination , so they're beyond the scope of this post. My second assumption is that you don't need help with steps 1-4 above. We're going to jump right into extracting cookies from Firefox and Chrome.

Update, 25-Apr-2010: Here's one last opportunity to stop reading and start doing. If you're a Firefox user, and not averse to using plugins in your browser, you could install a plugin like Cookie Exporter to extract cookies to a properly-formatted text file. Also, notwithstanding my crack about IE, I have read that it's possible to export IE's cookies in the correct format without having to resort to any special shenanigans. I had intended to point out both of these things in my original post, but instead I launched right into the SQLite discussion below. Sorry about that; the editing department has been sternly reprimanded.

Extracting cookies

Lynx very conveniently exports its cookies to a text file, in the format expected by wget (the classic "Netscape cookies.txt" spec). Firefox and Chrome, on the other hand, save cookies in a SQLite database file. So, I guess that's it, right? Cookies locked up in some goofy binary file. Not going to be able to get those out. Guess I'm out of luck.

Oh. Wait. I do database stuff. There's hope!

The first thing to do is to install a SQLite client. Instructions are beyond the scope of this post, but you can download precompiled binaries of the sqlite3 command-line client from the SQLite site. If you're a Mac user, you're in luck: sqlite3 should already be installed in /usr/bin.

The next step is to locate the cookies database file for your browser. For Firefox, it should be cookies.sqlite in your default profile directory, and for Chrome, it should be the file creatively named Cookies in your profile directory. If you're really stuck, and are running Linux or OS X, you should be able to find the location by issuing lsof | grep -i cookies while your browser is running.

Here's an example of what the cookies look like in the SQLite database in Firefox:

zathras:~ jpiwowar$ sqlite3 /Users/jpiwowar/Library/Application\ Support/Firefox/Profiles/ftpzvxjz.default/cookies.sqlite
SQLite version 3.6.10
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .schema moz_cookies
CREATE TABLE moz_cookies (id INTEGER PRIMARY KEY, name TEXT, value TEXT, host TEXT, path TEXT,expiry INTEGER, lastAccessed INTEGER, isSecure INTEGER, isHttpOnly INTEGER);
sqlite> select *
...> from moz_cookies
...> where name like 'ORA_UCM%'
...> ;
1271476327342462|ORA_UCM_INFO|3~70969please_dont_copy_my_cookies|.oracle.com|/|1303012327|1271477696515030|0|0
1271476327343576|ORA_UCM_VER|%200|.oracle.com|/|1303012327|1271477696515030|0|0
1271476327344121|ORA_UCM_SRVC|no_really_just_use_your_own|.oracle.com|/|1303012327|1271477696515030|0|0

The name of the cookies table in Firefox is moz_cookies. At the time of this post, the OTN cookies needed for this exercise have names starting with ORA_UCM. The name of the SQLite table in the Chrome cookies database is simply 'cookies'. You'll note that some of the column names are also different:

zathras:~ jpiwowar$ sqlite3 /Users/jpiwowar/Library/Application\ Support/Chromium/Default/Cookies
SQLite version 3.6.10
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .schema cookies
CREATE TABLE cookies (creation_utc INTEGER NOT NULL UNIQUE PRIMARY KEY,host_key TEXT NOT NULL,name TEXT NOT NULL,value TEXT NOT NULL,path TEXT NOT NULL,expires_utc INTEGER NOT NULL,secure INTEGER NOT NULL,httponly INTEGER NOT NULL,last_access_utc INTEGER NOT NULL);

Of course, a simple "select *" from the cookies table is not going to produce a usable cookies file to use with wget. The cookies.txt spec requires 7 tab-, not pipe-, separated fields, and the expiration timestamp for Chrome cookies is in an unusual format. Rather than take up even more space in this post, I've written a shell script, OTNcookies.sh (it'll probably download as .txt, not .sh), that performs the work of extracting the cookies in the proper format. Here's a quick overview of the script:

• Takes two arguments: the first (required) is the location of the cookies database file, and the second (optional) is the name of the text file containing the exported cookies.
• Determines the type of cookie database (Firefox or Chrome)
• Runs an appropriate query to extract the cookies to the text file specified by the user, or to a default output file.
• Tries to accommodate for spaces in filenames and directories, but you will need to escape spaces in the file names that you pass to the script

As with any script you grab from the Intertubes, please exercise caution. This isn't designed to do anything malicious or damaging, but that doesn't mean you couldn't do something silly with it (like, say, running it as root and overwriting critical system files with cookies. Wouldn't that be embarrassing? Yes, but it would be your fault, not mine )

Here's a sample run of the OTNcookies script:

zathras:~ jpiwowar$ OTNcookies.sh /Users/jpiwowar/Library/Application\ Support/Firefox/Profiles/ftpzvxjz.default/cookies.sqlite /var/tmp/FFcookies.txt
Looks like Firefox.
OTN cookies written to /var/tmp/FFcookies.txt

After extracting the cookies, it's a simple matter of invoking wget with the load-cookies option, as documented in Marc's article.

zathras:~ jpiwowar$ wget --load-cookies=/var/tmp/FFcookies.txt http://download.oracle.com/otn/mac/instantclient/10204/instantclient-basic-10.2.0.4.0-macosx-x86.zip

--2010-04-22 17:00:39--  http://download.oracle.com/otn/mac/instantclient/10204/instantclient-basic-10.2.0.4.0-macosx-x86.zip
Resolving download.oracle.com... 208.111.133.52, 208.111.133.54
Connecting to download.oracle.com|208.111.133.52|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 34109360 (33M) [application/zip]
Saving to: `instantclient-basic-10.2.0.4.0-macosx-x86.zip'<br />10% [===>                                   ] 3,563,808    545K/s  eta 65s

A final note: The cookie file should be re-usable, so you don't need to regenerate one for each file you want to download. The cookies will expire eventually, however, at the time indicated in the expiry or expires_utc field. At that point, you'll need to go through the whole process again, starting with logging in to OTN to create new cookies.

Happy downloading!

Here's one from the "learned something new today" files. When installing Oracle products on 64-bit versions of Linux, it's a common requirement to have both 32- and 64-bit versions of certain packages installed (that's not the "learned something new" part). The default output for the rpm query command, rpm -q, doesn't show the architecture of the queried package (that's not new, either). For example, installing Oracle RDBMS 11gR1 on RHEL4 or OEL4 requires both 32- and 64-bit versions of the libaio and glibc-devel (not the full list of package requirements; I'm just trying to keep the example short). A typical rpm query would show the following output:

[jpiwowar@testsrv ~]$ rpm -q libaio glibc-devel
libaio-0.3.105-2
glibc-devel-2.3.4-2.39
glibc-devel-2.3.4-2.39

It's probably safe to assume that those two glibc-devel entries cover the 32- and 64-bit versions, but where's the second libaio? The easy answer is to append the architecture string to the name of the package, but that results in a pretty quiet failure for the package that's not installed:

[jpiwowar@testsrv ~]$ rpm -q libaio.x86_64
libaio-0.3.105-2
[jpiwowar@testsrv ~]$ echo $?
0
[jpiwowar@testsrv ~]$ rpm -q libaio.i386
[jpiwowar@testsrv ~]$ echo $?
0

My new favorite trick (finally, the "learned something new" part!) is to use the --queryformat (or --qf) option to display the architecture of an installed package:

[jpiwowar@testsrv ~]$ rpm -q --queryformat '%{NAME}-%{VERSION}-%{RELEASE} %{ARCH}\n' libaio glibc-devel
libaio-0.3.105-2 x86_64
glibc-devel-2.3.4-2.39 i386
glibc-devel-2.3.4-2.39 x86_64

It's not perfect, but it makes it a bit easier to spot which version of a package (32-bit libaio, in this case) is missing in a long list of required packages. I'm a bit embarrassed to admit that I didn't know about this option until recently, but if it's new to me, maybe it's new to you too.

References

• The Parts of an RPM Query
• My Oracle Support Note 437123.1: Requirements for Installing Oracle 11gR1 RDBMS on RHEL 4 on AMD64/EM64T (just in case you landed here from a Google search for installing 11gR1 on RHEL4)

The excitement of solving mysteries makes me bad at math

In my last post, I failed to notice that a HugePages_Total of 4645, while satisfyingly greater than zero, is definitely not 8196, the desired and expected value I had configured. Upon reflection, the reason was pretty obvious: the OS will only allocate contiguous blocks of memory for hugepages, and the server had been up for long enough that memory was a bit fragmented. One reboot later, things were much better:

sles10db:~ # cat /proc/meminfo | grep -i huge
HugePages_Total:  8196
HugePages_Free:   8196
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Autoconfig giveth, and Autoconfig taketh away

When an E-Business Suite system is behaving in a way that suggests one or two mangled (some say "corrupt," but I try not to be judgmental) configuration files, a common first response is to run Autoconfig to see if the system can sort itself out. Running Autoconfig is also frequently necessary during the course of normal maintenance, e.g. during some patching, or when renaming/relocating Oracle Apps nodes. This weekend, I was reminded of an interesting side effect of running Autoconfig. After the process was complete, Grid Control started reporting metric collection errors and "Unknown" status for some components on the application tier nodes. In this case, Autoconfig was just doing its job, generating new versions of configuration files for the apps tier, complete with their original file permissions, which overwrote the permission changes I had made to enable the Grid Control agent to monitor some of the OC4J components. Good thing I keep that list of chmod commands in an easy-to-find reference location.

Oh yeah, and...

I also learned that using wireshark to try to debug a problem with a connection to an Oracle database over VPN is kind of like trying to listen to AM radio in a tunnel. You can tell there's signal there, but good luck making anything of it.

I feel like a n00b for even posting this, but if part of my goal here is to catalog my mistakes, well, here's one I probably won't make again.

I've been involved in a project to migrate an E-Business Suite database tier from 32-bit to 64-bit RDBMS. Thus far it's been pretty straightforward. One thing had been bugging me, though. Our original test platform was 64-bit SUSE Linux Enterprise Server 9 (SLES9), but a decision was made to upgrade to SLES10 instead. Not a big deal in this case, except for one thing: the settings I'd used in SLES9 to allocate hugepages weren't doing a darn thing in SLES10:

sles10db:~ # grep -i nr_huge /etc/sysctl.conf
vm.nr_hugepages = 8196

sles10db:~ # grep memlock /etc/security/limits.conf
oracle soft memlock 17000000
oracle hard memlock 17000000

Okay so far...

sles10db:~ # grep -i huge /proc/meminfo
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

sles10db:~ # cat /proc/sys/vm/nr_hugepages
0

Whahuh? Why wasn't my nr_hugepages setting getting picked up from /etc/sysctl.conf? Wrong question, of course. If other settings in that file were being picked up (they were), then clearly something else was clobbering the value I had set. Sure enough, after not much poking around, I found this:

sles10db:~ # grep -i nr_huge /etc/sysconfig/oracle
# /proc/sys/vm/nr_hugepages indicates the current number of configured hugetlb
NR_HUGE_PAGES=0

How embarrassing is that? It's even documented in a very official, check-here-first sort of place. Since /etc/init.d/oracle is run long after /etc/init.d/boot.sysctl, the values set in /etc/sysconfig/oracle clearly will clobber anything set in /etc/sysctl.conf. My only defense is that NR_HUGE_PAGES was not included in /etc/sysconfig/oracle on our SLES9 test system, which is why I was setting it in /etc/sysctl.conf in the first place. Of course, we might've been using an old version of orarun.rpm on that server, so it's still a pretty weak excuse. At any rate, changing the setting in /etc/sysconfig/oracle and running rcoracle restart did the job:

sles10db:~ # grep -i nr_huge /etc/sysconfig/oracle
# /proc/sys/vm/nr_hugepages indicates the current number of configured hugetlb
NR_HUGE_PAGES=8196

sles10db:~ # grep -i huge /proc/meminfo
HugePages_Total:  4645
HugePages_Free:   4645
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Hope this helps someone out there; otherwise, I'll just sit here, alone in my hopefully-temporary idiocy.

Finding the largest files in a directory on Linux is ridiculously easy, with the right flags to ls:

[oracle@vir demo]$ ls -1srShA | tail
4.0K jptest_mmnl_4899.trc
4.0K jptest_mmon_3011.trc
4.0K jptest_mmon_10186.trc
4.0K jptest_mmon_4897.trc
4.0K jptest_p001_3996.trc
4.0K jptest_p000_3994.trc
4.0K jptest_p000_3034.trc
4.0K jptest_p001_3036.trc
4.0K big directory with annoying name
104K alert_JPTEST.log

But hey, wait…what about that directory? It certainly seems to be conveniently named, almost as if placed there specifically for my follow-up question: What if I what I really want is to know the sizes of the largest files and subdirectories in my current directory? For that, I have to do something a bit more complicated than ls, but less complicated than writing a script:

[oracle@vir demo]$ topten
4.0K    ./jptest_mmnl_4899.trc
4.0K    ./jptest_mmon_10186.trc
4.0K    ./jptest_mmon_3011.trc
4.0K    ./jptest_mmon_4897.trc
4.0K    ./jptest_p000_3034.trc
4.0K    ./jptest_p000_3994.trc
4.0K    ./jptest_p001_3036.trc
4.0K    ./jptest_p001_3996.trc
104K    ./alert_JPTEST.log
529M    ./big directory with annoying name

topten is a shell function defined as follows (you'll probably need to scroll a bit):

[oracle@vir demo]$ type topten
topten is a function
topten ()
{
    find . -maxdepth 1 -not -name '.' -print0 | xargs -0 du -s | sort -n | tail | cut -f2- | awk '{ printf("%s%c", $0, 0) }' | xargs -0 du -sh
}

One could claim that this is ridiculously over-engineered, and I would sheepishly agree. In fact, sharing this on a blog is my only hope of recouping the time spent in constructing the function in the first place. After all, what’s wrong with something as simple as du -s * | sort -n | tail? Well, starting with item #1, below, things just sort of snowball:

1. In directories with a lot of files, du -s fails with a “too many arguments” error. $APPLCSF/log on an E-Business Suite midtier springs to mind.
2. Using xargs solves the "too many arguments" problem, but introduces a new one: xargs will misinterpret whitespace in file names as an argument separator, and split the file name. Whitespace in file names? Hey, I work on a Mac. This stuff happens. Users of Cygwin on Windows probably get bitten by this, too.
3. The use of find -print0 | xargs -0 is a common solution to the "whitespace in filenames" issue.
4. I like the human-readable output from du -h, but that means I need to pass my top ten files through xargs again, which requires more filtering: cut removes the leading numbers from the output of the first du, and the awk command provides null-terminated output like find's -print0.

Caveats and variations

• Presumably, this function should work in Bourne-shell variants on any Unix-like system (works fine on Linux and Mac, for example). If you aren't running the GNU versions of the utilities invoked in this function, however, you may need to make some adjustments.
• It can be expensive to use du. Choose your starting directory with care. If you run this command in the E-Business Suite APPL_TOP, for example, you might want to find something else to do for a few minutes. If you run this command on an already-busy production system, you may also want to find a place to hide from your sysadmins.
• If you don’t care about the prettiness of the human-readable file sizes, then you can shorten the function to omit everything after the tail command:

  topten ()
  {
  find . -maxdepth 1 -not -name '.' -print0 | xargs -0 du -s | sort -n | tail
  }
  

• The function is called topten because the default output for the tail command is 10 lines. Provide the appropriate option to tail to change the output length (tail -5 for 5, tail -20 for 20, etc.)
• It may be advantageous to add the -x option to du to prevent it from crossing filesystem boundaries.
• There's probably an easier, and still portable, way to do this. I'm kinda hoping that someone stops by with a comment to tell me what that is.

Update #1, 13-Nov-2009: While testing this patch download method in the wake of the recent My Oracle Support cutover, I noticed a new wrinkle. Instead of a successful patch download, I got the following message from wget:

Self-signed certificate encountered.
To connect to login.oracle.com insecurely, use `--no-check-certificate'.

After adding --no-check-certificate to the wget command, I was once again able to retrieve patches, but it took a bit longer than usual, as wget wandered through a tree of redirects. I've updated the code for the getOraPatch function accordingly.

Update #2, also 13-Nov-2009: A commenter on Chris Warticki's Oracle Support blog noted that the patch download page provided by the Flash version of My Oracle Support does not present a right-clickable Download button, which would seem to render the wget patch retreival method useless. Thankfully, there's a workaround: open a new window/tab in your browser, using an URL of the form https://updates.oracle.com/ARULink/PatchDetails/process_form?patch_num=nnnnnnn This will open a patch download page in a plain HTML, non-Flashy interface, and you should be all set to go!

No, no, this isn't a back door into the Metalink My Oracle Support patch repository. Don't get too excited.

While working from home this evening, I had to download some patches to a remote Linux system, and was struck, not for the the first time, by the inconvenience of it all. The usual options seem to be:

1. Download the patches to a local machine and upload them to the remote system -- annoyingly manual at best, and downright gross when a VPN connection is involved.
2. FTP from the remote system to updates.oracle.com. That's a tried-and true method, but still involves a lot of typing. Not anymore! This access option was disabled in November 2009, with the arrival of My Oracle Support.
3. Start a VNC session on the remote system, launch a web browser, log in to Metalink My Oracle Support, and click the download button. Not a bad option...unless you can't use VNC for some reason, and are stuck with a command-line interface. Or if you're trying to navigate in that remote browser over a VPN connection. Not an exercise for impatient people.

When life gives you inconvenience, write a shell function!

At least, I think that's how the old saying goes. I created the following bash(1) function that uses wget to download a patch from updates.oracle.com, using the URL behind the Download button:

# Function to retreive patches from Metalink/My Oracle Support via wget
function getOraPatch {
   [[ $mosUser ]] || read -p "Oracle Support Userid: " mosUser;
   [[ $mosPass ]] || read -sp "Oracle Support Password: " mosPass;
   fname=`echo $1 | awk -F"=" '{print $NF;}'`;
   wget --no-check-certificate --http-user $mosUser --http-passwd $mosPass $1 -O $fname;
}

The above function prompts for the username and password that you use to access Metalink My Oracle Support. If the variables are already defined in your session, then it assumes you want to use the existing values and does not prompt you again. Then, the function parses the patch URL to find the patch's filename, and passes all four values (userid, password, URL, and filename) to wget.

Usage example

1. Load the above function into your shell session, or copy and paste the code into your .bash_profile to have it loaded when you log in.
2. Log in to Metalink My Oracle Support and locate the patch. Can't get around this step, but you can do it in a local browser, instead of dealing with a remote X session as described above in #3.
3. Right-click on the Download button to copy the patch URL:
   
4. Invoke the function, enclosing the super-long patch URL in single quotes:

   oracle@testhost:/stage> getOraPatch 'really_long_patch_URL_copied_in_previous_step'
   

5. Answer the prompts, and lean back and watch as wget retrieves your patch:

   Oracle Support Userid: your_MyOracleSupport_login_here
   Oracle Support Password: your_MyOracleSupport_password_here
   (some wget output snipped)
   HTTP request sent, awaiting response... 200 OK
   Length: 52,028 [application/zip]
   
   100%[====================================>] 52,028        --.--K/s             
   
   22:27:24 (555.94 KB/s) - `pnnnnnnn_version_platform.zip' saved [52028/52028]
   

6. Repeat as necessary.

Disclaimers, caveats and commentary

1. As noted in the opening line of this post, this is not a "back door" into Oracle's patch repository. You still need a valid Metalink userid and password for this retrieval method to work. Furthermore, since this method essentially performs the same actions as clicking on a link in a browser, and requires manual collection of the patch download URL, I don't think it qualifies as "automation." I certainly didn't set out to violate the Oracle.com Terms of Use, so hopefully no one interprets this as such.
2. I've only tested this on Linux. Presumably, you could utilize this method on other Unix variants (and even Windows, if you're into that sort of thing).
3. On Unix/Linux, make sure you enclose the patch URL in single quotes, or the shell will interpret all of the ampersands in the URL as, "wow, you want me to launch 5 or 6 background processes? O-kay!"
4. Before copying the patch URL, make sure you've already selected the correct release and platform for the patch. The values of the aru and file_id parameters in the URL change as you select those values.
5. You need to have wget installed on your system for this method to work as written above. It probably would be relatively easy to adapt this to use curl instead, but wget is what I thought to use first.
6. Some versions of wget use http-password instead of http-passwd. Check wget --help to verify what's valid for your version.
7. Using this function will leave the values of your Oracle Support userid and password in the shell variables $mosUser and $mosPass. If this makes you uncomfortable, remember to use unset on those variables, or just exit your session, when you're done downloading.
8. If you have any characters in your password that could be interpreted as metacharacters by the shell, you might need to escape them. I didn't test that too thoroughly.

A few random observations

• Yes, I'm downloading patches on a Friday night. And it's not urgent, strictly speaking. Do I know how to live, or what?
• According to The Google, my posting rate for the life of this blog is 0.2 posts/week. It's not every week that I get to surpass my average output by an order of magnitude.
• On second thought, at this rate, I could post twice a week for a while and still bask in the glow of that false feeling of productivity.

When installing Grid Control agents on a Linux or Unix platform, a recommended practice is to install as a user that doesn't own the ORACLE_HOMEs to be monitored. This poses a challenge when configuring monitoring for some targets, particularly those based on Oracle Application Server 10g. For security purposes, file and directory permissions on some ORACLE_HOMEs are fairly restrictive, and the Grid Control agent can't read all of the files necessary for target discovery and metric collection. The Grid Control release notes mention this complication, and Metalink Note 437078.1 provides additional suggestions for resolving discovery and metric collection errors related to file permission restrictions on a target ORACLE_HOME. I can't reveal the content of the Metalink My Oracle Support Note, and the information in the publicly-available Release Notes provides inadequate coverage, but the overall problem-solving method reduces to:

1. Make sure that the owner of the agent software is in the same group as the owner of the monitored target software
2. Check the agent log files for file permission errors, and fix them.

My experience so far indicates that fixing some errors tends to reveal others. Who doesn't love a rousing game of log file whack-a-mole? In the interest of saving others a few rounds of this, I'm offering up a few additional permission change steps that could make your lives easier. For ease of navigation, I've broken things into the following sections:

• Oracle Application Server
• Database targets
• Oracle Collaboration Suite Calendar Server
• E-Business Suite

Disclaimer: When considering making the changes I describe below, please consider the trade-off you are making: more robust monitoring at the expense of some level of security. The perceived degree of sacrifice will vary from person to person and organization to organization. I would argue that the benefits outweigh the risks, and since Oracle itself suggests similar changes, this is clearly not a cut-and-dried issue. Hey, if doing the right thing were always easy, this job wouldn't be any fun, right?

Please feel free to leave any similar tips (or links to your own posts about this topic) in the comments.

Oracle Application Server

Note 437078.1 is a good start, but there are a few additional items I found to ease metric collection and discovery errors for Oracle 10g App Server. ORACLE_HOME refers to the location of the 10gAS software. Depending on the application server components installed on your system, you may need to change more or fewer permissions.

chmod g+rx $ORACLE_HOME/webcache
chmod g+r $ORACLE_HOME/webcache/*.xml
chmod g+rx $ORACLE_HOME/portal
chmod g+rx $ORACLE_HOME/portal/conf
chmod g+rx $ORACLE_HOME/discoverer
chmod g+rx $ORACLE_HOME/discoverer/config
chmod g+rx $ORACLE_HOME/forms
chmod g+rx $ORACLE_HOME/forms/server
chmod g+rx $ORACLE_HOME/ldap
chmod g+rx $ORACLE_HOME/ldap/das
chmod g+rx $ORACLE_HOME/uix
chmod g+rx $ORACLE_HOME/ultrasearch
chmod g+rx $ORACLE_HOME/ultrasearch/webapp
chmod g+rx $ORACLE_HOME/ultrasearch/webapp/config
chmod g+rx $ORACLE_HOME/jpi
chmod g+rx $ORACLE_HOME/jpi/doc
chmod g+rx $ORACLE_HOME/Apache/oradav
chmod g+rx $ORACLE_HOME/Apache/oradav/conf
chmod g+rx $ORACLE_HOME/Apache/jsp
chmod g+rx $ORACLE_HOME/Apache/jsp/conf
chmod g+rx $ORACLE_HOME/Apache/modplsql
chmod g+rx $ORACLE_HOME/Apache/modplsql/conf
chmod g+rx $ORACLE_HOME/sso
chmod g+rx $ORACLE_HOME/sso/conf

Database targets

Some environments (for example, the 10gAS infrastructure database) will require permission changes to effectively monitor database and listener targets. You may find the scope of these changes to be a bit broad; at some point I got tired of chasing down individual files and just hit them all with the same hammer. In this case, ORACLE_HOME refers to the ORACLE_HOME of the RDBMS software.

find $ORACLE_HOME/admin -type d -exec chmod g+rx {} \;
find $ORACLE_HOME/admin -type f -exec chmod g+r {} \;

find $ORACLE_HOME/network -type d -exec chmod g+rx {} \;
find $ORACLE_HOME/network -type f -exec chmod g+r {} \;

find $ORACLE_HOME/rdbms -type d -exec chmod g+rx {} \;
find $ORACLE_HOME/rdbms -type f -exec chmod g+r {} \;

## The following are needed for database health checks to work properly...
chmod g+rx $ORACLE_HOME/dbs
chmod g+rw $ORACLE_HOME/dbs/hc*

Another general configuration note for the database listener, which I stumbled upon when setting up monitoring for a Secure Enterprise Search repository database: If sqlnet.ora is configured with TCP.VALIDNODE_CHECKING=yes, then the server that hosts the Grid Control OMS needs to be added to the list of hosts defined by TCP.INVITED_NODES. Otherwise, you'll get messages like this when attempting to add the database target, even if all monitoring credentials are correct:

Failed to connect to the database: Io exception: Got minus one from a read call
The Connect Descriptor was (description=(address=(host=ses.myorg.com)(protocol=tcp)(port=xxxx))
(connect_data=(service_name=SESdb)(instance_name=SESdb)(UR=A)))</pre>
And corresponding messages in the listener log:
<pre>TNS-12546: TNS:permission denied
 TNS-12560: TNS:protocol adapter error
  TNS-00516: Permission denied

Of course, after adding the OMS host to TCP.INVITED_NODES in sqlnet.ora, you need to tell the listener that you've made a change before the OMS will be able to connect to the database and add it as a target:

oracle@ses:~> lsnrctl  reload

LSNRCTL for Linux: Version 10.1.0.5.0 - Production on 05-DEC-2008 10:58:45

Copyright (c) 1991, 2004, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=ses.myorg.com)(PORT=xxxx)))
The command completed successfully

Oracle Collaboration Suite Calendar Server

These commands will resolve some issues with metric collection for OCS Calendar Server. I suspect that the intersection of Collaboration Suite administrators (a small set) and readers of this blog (a vanishingly small set) is probably zero, but Google works in mysterious ways. ORACLE_HOME refers to the OCS software location, not the Grid Control agent home.

chmod g+rx $ORACLE_HOME/ocal/bin
chmod g+rx $ORACLE_HOME/ocal/bin/uniwho
chmod g+rx $ORACLE_HOME/ocal/bin/unireqdump
chmod g+rx $ORACLE_HOME/ocal/bin/uniping
chmod g+rx $ORACLE_HOME/ocal/bin/unistatus
chmod g+rx $ORACLE_HOME/ocal/sbin
chmod g+rx $ORACLE_HOME/ocal/sbin/who
chmod g+rx $ORACLE_HOME/ocal/sbin/reqdump
chmod g+rx $ORACLE_HOME/ocal/sbin/ping
chmod g+rx $ORACLE_HOME/ocal/sbin/status
chmod g+rx $ORACLE_HOME/ocal/oem
chmod g+rx $ORACLE_HOME/ocal/oem/scripts
chmod g+rx $ORACLE_HOME/ocal/oem/scripts/ocal_ps.pl
chmod g+rx $ORACLE_HOME/ocal/oem/scripts/ocal_dbsize.pl
chmod g+rx $ORACLE_HOME/ocas
chmod g+rx $ORACLE_HOME/ocas/linkdb
chmod g+rx $ORACLE_HOME/ocas/sessiondb
chmod g+rx $ORACLE_HOME/lib
chmod g+rx $ORACLE_HOME/ocal/lib
chmod g+r $ORACLE_HOME/ocal/lib/*.so
chmod g+rx $ORACLE_HOME/ocal/db
chmod g+rx $ORACLE_HOME/ocal/db/nodes
find $ORACLE_HOME/ocal/db/nodes -type d -exec chmod g+rx {} \;
find $ORACLE_HOME/ocal/db/nodes -type f -exec chmod g+r {} \;

E-Business Suite

Setting these permissions in an E-Business Suite R12 instance resolved some metric collection errors of the form

Couldn't open INST_TOP/ora/10.1.3/j2ee/oafm/application-deployments/oafm/orion-application.xml: Permission denied at AGENT_HOME/sysman/admin/scripts/ias/simpleXPath.pm line 116

These errors may not manifest in 11i, since that version's tech stack is not based on 10gAS. INST_TOP is the "Instance TOP" environment variable for the R12 Apps installation, and serves as a reminder to connect as the Apps owner (not database software owner) to run these commands.

chmod g+r $INST_TOP/ora/10.1.3/j2ee/forms/config/jms.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/forms/application-deployments/forms/orion-application.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/forms/application-deployments/forms/formsweb/orion-web.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oacore/config/jms.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oacore/application-deployments/oacore/orion-application.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oacore/application-deployments/oacore/html/orion-web.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oafm/config/jms.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oafm/application-deployments/oafm/orion-application.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oafm/application-deployments/oafm/webservices/orion-web.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oafm/application-deployments/mapviewer/orion-application.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oafm/application-deployments/mapviewer/web/orion-web.xml
chmod g+r $INST_TOP/ora/10.1.3/j2ee/oafm/application-deployments/ascontrol/orion-application.xml

I've been involved in a project to deploy Oracle Grid Control recently, and thought I would share my list of reference documents. Even for a relatively simple deployment, there's a lot of reading to be done. I'm sure this surprises no one. I don't claim to have compiled an exhaustive list of documents and patches. Depending on your platform and deployment architecture, you may have more or less (probably more) research to do. Nevertheless, if you're in the early stages of planning a Grid Control deployment project, perhaps this link collection will help you get started. As time goes on, these references may become less relevant. Patches in particular can have a ridiculously short half-life, but at least the Metalink My Oracle Support documents tend to be updated in place rather than superseded.

Before I launch into the long list, here's a high-level view of the system I've been deploying, to help you determine if your project is more or less (again, probably more) complicated than mine:

• 2-node Grid Control system (1 repository, 1 OMS)
• Linux x86 for repository and OMS
• Repository database version 10.2.0.4
• OMS and agent versions 10.2.0.4 (sometimes referred to as Grid Control 10gR4)
• Small-ish (less than 500) set of monitored targets
• Mix of Linux x86, Windows, and Linux x86-64 targets, predominantly Linux x86
• Some E-Business Suite targets, with Applications Management Pack licensed

Metalink Reference Documents

Database

Note 555577.1: Oracle Database Patch Set Notes 10g Release 2 (10.2.0.4) Patch Set 3 for Linux x86

A good, quick read for the 10.2.0.1->10.2.0.4 upgrade.

Note 359145.1: Impact of 2007 USA daylight saving changes on the Oracle database

According to the matrix in this note, RDBMS version 10.2.0.4 contains the necessary fixes for current US DST, so no additional patching required for the repository database

Note 735216.1: Critical Patch Update October 2008 Availability Information for Oracle Database and Fusion Middleware Products

Also applicable to Application Tier components, as the title suggests.

E-Business Suite/AMP

Note 394448.1: Getting Started with the Application Management Pack for Oracle E-Business Suite (Releases 2.0 and 2.0.1)

A short overview document, most useful for its links to the Oracle Application Management Pack for Oracle E-Business Suite Installation Guide, Oracle Application Management Pack for Oracle E-Business Suite User's Guide

Note 412044.1: Application Management Pack for Oracle E-Business Suite Release 2.0 and 2.0.1 Troubleshooting Guide

Probably won't need this, since most deployments go completely smoothly and troubleshooting is rarely required, right?

Note 406982.1: Cloning Oracle Applications Release 12 with Rapid Clone

AMP installation requires the latest Rapid Clone patches. This is a quick way to get links to them. If you're running 11i instead of R12, be sure to review the relevant document for that release.

Grid Control

Note 412431.1: Oracle Enterprise Manager 10g Grid Control Certification Checker

You could, of course, just use the standard certification matrix to figure out what versions of the Grid Control components are certified for your platform, but this document is much more concise, not to mention prettier.

Note 457792.1: DST Compliance for Grid Control OMS, Repository and Agents: Checks and Updates

Note 409225.1: Impact of DST changes on Grid Control Release 2 - Preparation and Patching guidelines

Note 471451.1: DST 2007/2008 Oracore files Version for Grid Control Components

The preceding three notes will help to untangle the Daylight Savings Time patching requirements for the various components of Grid Control. The highly-compressed bottom line: Getting all the components to version 10.2.0.4 will save a lot of pain.

Note 468959.1: Enterprise Manager Grid Control Known Issues

Note 602750.1: ConfigureGC.pl Reports - Invalid Username/Password.

I got bitten by this issue when performing the software-only OMS installation.

Note 578715.1: How to Install Grid Control 10.2.0.4.0 with a New Database using the Software-only Option

Useful reference for "existing database" option as well, as as a supplement to the installation docs.

Note 414989.1: How to Fix Generic_MOM_Managed_Host In Pending Status?

The Note refers to OMS version 10.2.0.3, but I also encountered this issue when deploying version 10.2.0.4

Note 437078.1: Problem : iAS Discovery/Monitoring Fails If Agent Is Installed Under Another OS User Than iAS

This topic is also addressed in the Grid Control release notes, but this Note is more comprehensive.

Application Server 10g

Note 566418.1: How to Recover OUI to 10.2.0.3/10.2.0.4 after Applying IAS 10.1.2.2 Patchset on Grid Control OMS Home

Upgrading the OMS IAS software to 10.1.2.2 is necessary to meet the minimum version requirements for the latest CPU patches. Unfortunately, performing that upgrade downgrades the Oracle Universal Installer in the OMS ORACLE_HOME. Whoopsie. This Note describes how to fix it.

Note 329361.1: Oracle Application Server 10g Release 2 (10.1.2) Support Status and Alerts

Useful reference for finding 10gAS patchsets.

Note 415222.1: Steps to Maintain Oracle Application Server 10g Release 2 (10.1.2)

Relevant for upgrading OMS ORACLE_HOME to 10.1.2.2. Reference "Step 3: Apply Latest Oracle Application Server 10.1.2.2 Patchset (Patch 4960210)."

Note 280116.1: How To Reset Ias_admin User Password On AS10G On Unix If The Old Password Is Missing?

...or, "if you can't be bothered to look up the value of the default password after a silent install, since you're just going to change it anyway."

Note 270516.1: How To Change The SYSMAN Password Used For Grid Control

This information can also be found in the Installation and Configuration Guide on OTN (see below).

Note 436299.1: Common reasons for OUI failures on the MS Windows platforms

I work primarily on Linux, but every now and then, I get an "opportunity" to install Oracle software on Windows. The little issues listed in this Note are worth reviewing before installing a Grid Control monitoring agent on Windows.

Non-Metalink references

Oracle® Enterprise Manager Grid Control Release Notes for Linux and Microsoft Windows 10g Release 4 (10.2.0.4.0)

Although the release notes state that RDBMS 10.2.0.4 is not a certified repository version, the more-up-to-date Oracle Enterprise Manager 10g Grid Control Certification Checker refutes this.

Oracle Enterprise Manager Grid Control Installation and Basic Configuration 10g Release 4 (10.2.0.4)

Oracle Enterprise Manager Advanced Configuration 10g Release 4 (10.2.0.4.0)

See Section 5.3 for configuring integration w/ Single Sign-On

10g R2 Management Agent Deployment Best Practices

Patches

Patches tend to be superseded fairly rapidly, CPUs in particular. The following are presented primarily to provide an idea for the amount of patching to expect. Be sure to check the documentation to verify relevance.

E-Business Suite

6809246: APPLICATIONS MANAGEMENT PACK V2.0.2 UPDATE FOR E-BUSINESS SUITE

5489352: EM PLUGIN V2 FOR ORACLE APPLICATIONS

6510214: R12.AD.A.DELTA.4

6776948: R12 RAPIDCLONE CONSOLIDATED FIXES JUL/2008

6874927: R12 Interoperability Patch for AMP 2.0.2

Grid Control repository database

6810189: 10.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER

6880880: Universal Installer: Patch OPatch 10.2

Be sure to download the 10.2 version!

7375644: MLR BUG FOR 10.2.0.4 FOR CPUOCT2008

Oct2008 CPU for Repository database

Grid Control management server

4700543: SSL SESSIONS FAIL DURING NEGOTIATION

Network client on OMS server, per Note 415222.1

4960210: PLACEHOLDER BUG FOR AS/DS 10G R2 PATCH SET 2 10.1.2.2

At this time, Version 10.1.2.2 is the maximum 10gAS release supported for the Grid Control OMS, and the minimum supported for application of the Oct2008 CPU patchset.

6342108: EBS DASHBOARDS FAIL TO RENDER, GENERATE STACK IN OC4J LOG

Be sure to grab 10.2.0.4 version

5861907: IAS 10.1.2.2 PATCHSET UPDATES ORACLEHOMEPROPERTIES.XML WITH WRONG ARU_ID & ARU_I

Required to enable opatch to apply additional patches once 10gAS ORACLE_HOME has been updated to 10.1.2.2

2617419: OPATCH FOR EM

For patching OMS ORACLE_HOME. Select 10.1.0.2 release when downloading

7040389: S/W ONLY AND CONFIG LATER METHOD- WEBSITE TARGET NOT CONFIG AND ONLINE HELP

6640752: OUI CD PLACE HOLDER ON ARU FOR 10204

Whoopsies. Check Note 566418.1 for details

7379121: MLR ON TOP OF 10.1.2.2 FOR CPUOCT2008

Oct2008 CPU for OMS tier

6606233: PERF: INEFFICIENT QUERY TO CALCULATE TOTAL NUMBER OF NEW POLICY VIOLATIONS

Be sure to download 10.2.0.4 version.

3731593: PATCHSETS FOR ENTERPRISE MANAGER GRID CONTROL

Grid Control agent

6670069: GETTING INITIALIZATION ERROR WHEN RUNNING FILETRANSFER COMMAND ITERATIVELY

When working on servers with multiple ORACLE_HOMEs, keeping a session's Oracle-related environment variables in alignment is critical to an administrator's sanity. Different versions of the RDBMS or application server software can require different binaries and shared libraries, and different ORACLE_HOMEs may have different network setups. An improperly configured Oracle environment can lead to TNS errors when attempting to connect to a database ("Oh, it's trying to use that tnsnames.ora..."), restarts of the wrong app server ("Wait a minute, which opmnctl did I just run?"), or wasted weekend afternoons (I once sat on the phone w/ Oracle Support for 3 hours because $ORACLE_HOME/ctx/lib was not in my LD_LIBRARY_PATH. Good times!).

Oracle addresses the need for managing environment configuration with the 'oraenv' script, and with the various environment files that need to be sourced to work in the different contexts of Oracle Applications. Furthermore, a number of common utilities on the application server side of the world, such as emctl and opmnctl, are shell scripts that explicitly set the Unix environment before launching processes that do actual work. For some systems, however, these tools may not be enough. For example, the oraenv script does not set TNS_ADMIN, and does not remove any custom additions to the PATH environment variables that are based on a previous value of ORACLE_HOME.

Some simple solutions

There are a number of simple ways to prevent your Unix session environment from becoming a tangled mess of inappropriate paths, without resorting to setting individual environment variables manually, including:

• Don't switch Oracle environments during a session. Instead, always start a new session to work in a new environment. This is can be good for maintaining personal discipline, but a bit tedious in practice, and besides, if I stopped there I wouldn't have much of a blog post, would I?
• Create separate shell scripts to do the work of setting up the environment variables needed, appending or prepending Oracle-related values. This is a common practice, but after switching back and forth between environments a few times, the PATH environment variable can grow from this:

  /u01/app/oracle/product/client/10gR2/bin:/u01/app/oracle/product/client/10gR2/OPatch:
  /u01/app/oracle/product/client/10gR2/opmn/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:
  /opt/oracle/bin

  ...to something like this admittedly contrived monstrosity:

  /u01/app/oracle/product/client/11gR1/bin:/u01/app/oracle/product/client/11gR1/OPatch:
  /u01/app/oracle/product/client/11gR1/opmn/bin:/u01/app/oracle/product/client/10gR2/bin:/u01
  /app/oracle/product/client/10gR2/OPatch:/u01/app/oracle/product/client/10gR2/opmn/bin:/u01/app
  /oracle/product/client/11gR1/bin:/u01/app/oracle/product/client/11gR1/OPatch:/u01/app/oracle/product
  /client/11gR1/opmn/bin:/u01/app/oracle/product/client/10gR2/bin:/u01/app/oracle/product/client
  /10gR2/OPatch:/u01/app/oracle/product/client/10gR2/opmn/bin:/usr/kerberos/bin:/usr/local/bin:/bin:
  /usr/bin:/opt/oracle/bin

  Alternately, instead of appending/prepending values to the environment variables, you could just reset them to default values, but those defaults may vary across servers, which can decrease portability of your scripts.

• Unset all Oracle-related environment variables manually before setting up the new Oracle environment. This works well for things like ORACLE_HOME and TNS_ADMIN, but what about PATH and LD_LIBRARY_PATH? Doing 'unset PATH' is a surefire way to wreck a perfectly good session, and resetting PATH by copying only the elements you wish to preserve can be error-prone.

Another solution

I'd like to have a method to unset just the Oracle environment in my session; that is, it should remove references to the current ORACLE_HOME from the environment and leave other environment variables untouched. What I've settled on is the code shown below, which can be adapted into a shell function, dropped into an existing environment setup script, or placed in a standalone script:

#!/bin/bash
#Remove references to current ORACLE_HOME in various environment variables.
#Do nothing if ORACLE_HOME is not set

if [[ -n $ORACLE_HOME ]]
then
   echo "Removing ORACLE_HOME references from environment"
   echo "Old ORACLE_HOME: $ORACLE_HOME"

   export LD_LIBRARY_PATH=`echo $LD_LIBRARY_PATH | sed "s#$ORACLE_HOME[^:]*:*##g"`
   # Include additional lines for clearing CLASSPATH, PERL5LIB, etc. as
   # needed *before* modifying PATH

   export PATH=`echo $PATH | sed "s#$ORACLE_HOME[^:]*:*##g"`
   unset TNS_ADMIN ORACLE_HOME ORACLE_SID ORACLE_BASE
fi

Here's a demo:

[oracle@lyta ~]$ . setup_11gR1.env
[oracle@lyta ~]$ echo $ORACLE_HOME; echo $LD_LIBRARY_PATH; echo $PATH
/u01/app/oracle/product/client/11gR1
/u01/app/oracle/product/client/11gR1/lib
/u01/app/oracle/product/client/11gR1/bin:/u01/app/oracle/product/client/11gR1/OPatch:/u01/app/oracle/product/client/11gR1/opmn/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/opt/oracle/bin
[oracle@lyta ~]$ . clear_oracle
Removing ORACLE_HOME references from environment
Old ORACLE_HOME: /u01/app/oracle/product/client/11gR1
[oracle@lyta ~]$ echo $ORACLE_HOME; echo $LD_LIBRARY_PATH; echo $PATH

/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/opt/oracle/bin

Since one person's completely sensible regular expression can be another person's incomprehensible line noise, here are some notes on the regex used in the sed command:

• Uses # instead of the traditional / as a separator to prevent errors resulting from interpolation of the $ORACLE_HOME environment variable.
• [^:]* matches zero or more non-colon characters. Since the colon is the separator for entries in the path, this prevents the regex from matching more than one entry at a time.
• :* removes the colon from the end of a matched path entry, so the revised path doesn't contain a string of useless separators (:::::::).

I may have just presented a solution in search of problem. On the other hand, I've found this to be useful, so maybe you will, too. If so, please leave a glowingly grateful comment. If not, then I've just revealed myself to the entire Internet as a slightly obsessive geek. There's a first time for everything.

Here's the second part of the discussion begun in my last post. No big reason for the split, other than putting both parts together made the post feel too long, and at some point laziness prevailed and I couldn't bring myself to edit things down any more.

Directory shortcuts

I define the following environment variables after sourcing the Apps environment file. These shortcuts allow me to navigate to frequently-used directories a bit faster. Not surprisingly, they relate to concurrent managers and patching:

export CM_LOG=$APPLCSF/$APPLLOG
export CM_OUT=$APPLCSF/$APPLOUT
export PATCH_BASE=/opt/oracle/stage/patch
export EBS_PATCH=$PATCH_BASE/ebs
export PATCH_LOG=$APPL_TOP/admin/$TWO_TASK/log
export WORKER_LOG=$PATCH_LOG

Yes, those last two are duplicates, which is more an indicator of my laziness (um, I mean, reluctance to context-switch unnecessarily) than anything else.

More fun with aliases: adpatch and adctrl

The following aliases save some typing at patch time. The first invokes adpatch in a the typical way, with the benefit of not having to answer all of the startup prompts. The second, as the name of the alias suggests, allows "hot patching" for those really small patches when you don't want to put the system in maintenance mode (only for use in test systems, though, right? Of course right).

alias patch_this='adpatch patchtop=`pwd` driver=`ls u*.drv` logfile=`ls u*.drv` defaultsfile=$DEFAULTSFILE workers=$WORKERS'
alias hotpatch_this='patch_this options=hotpatch'

A few things to note about using these adpatch aliases:

• It is necessary to be in the directory containing the patch driver before invoking one of the aliases, for example:

  	applr12@jpr12:~> cd $EBS_PATCH/6510214
  	applr12@jpr12:/opt/oracle/stage/patch/ebs/6510214> patch_this

• The aliases will only work on patches with 'universal' drivers, not the old-style individual copy, generate, and database patch drivers.
• The patch log file generated by using the aliases will have a .drv extension instead of .log. I've elected to live with the limitation rather than make the alias more complicated.
• Finally, it's necessary to define a WORKERS variable prior to using the aliases, either by exporting it in the .profile or defining it in an individual session.

During the patching process, the following aliases help to monitor and manage workers via adctrl:

alias adctrl_base='adctrl defaultsfile=$DEFAULTSFILE'
alias show_workers='adctrl_base menu_option=SHOW_STATUS'
alias start_addist='adctrl_base distributed=y worker_range=$WORKER_RANGE'

Update 04 Dec 2008: Lon White over at the Triora Group has a great (and more detailed) description of automated patching that is worth reading if you're unfamiliar with the idea of running adpatch non-interactively.

Interactive or non-interactive?

When setting up aliases for the AD utilities, I avoid using the interactive=n option. It's a useful option in the right circumstances, but I do not place it in my aliases by default, because it interferes with my primary goal in creating the shortcuts: typing less. Consider the following, after an AutoConfig run that reset the value of the APPLSYS password in the defaults file:

applr12@jpr12:~> adadmin defaultsfile=$DEFAULTSFILE menu_option=DISABLE_MAINT_MODE interactive=n
(adadmin startup stuff snipped, until...)
***
Enter the ORACLE password of Application Object Library :
***

AD Administration could not find a response to the above prompt
or found an incorrect response in the defaults file.

If I had run adadmin in normal, interactive mode, I would have been given a chance to enter the correct password the first time around. Instead, I now have to re-run adadmin in interactive mode to provide the password. More typing, not less. Bah!

Closing notes and annoying preachiness

• I've called this short series of posts is "Enhance your applmgr .profile," but it also could have been titled "Fun with AD utility command-line options." Refer to the Oracle Applications Maintenance Utilities guides for 11i and R12 for more information, or run the various commands (adadmin, adpatch, adctrl) with the help=y option to see what's possible.
• I've compiled a sample applmgr .profile that incorporates the ideas presented in these posts. Feel free to copy it and revise as necessary to suit your environment.
• And now for the preachy part: As fun as it can be to devise shortcuts for your favorite administrative tasks, resist the temptation to overdo it. Remember, this is primarily about saving time and reducing errors. For example, while having an alias to bounce Apache services can be useful, assigning an alias to adstpall.sh is probably overkill.