Enkitec E4 2013

Just a quick note that I'll be presenting at this year's Enkitec E4 conference. You can find the schedule here. I did some under the hood investigation regarding how the whole DBFS stack works from the performance perspective and, needless to say, some findings simply left me startled. If you want to see a session which will forever change the way you look at DBFS then this will definitely be the one to attend.

Oracle GoldenGate Integrated Capture #2

I have already written some words on the subject before. However, since then some interesting things have happened.

To recap (or save you some time if you don't want to read the original article) GoldenGate Integrated Capture is nothing else but Oracle Streams Capture in disguise. When running in the Integrated Capture mode pretty much the entire GoldenGate front-end gets yanked out of the picture and replaced with Oracle Streams Capture technology instead.

Let's take a closer look at some of the differences that happened between now and then. Here are two capture processes:

SQL> select capture_name, rule_set_name, purpose from dba_capture order by 1;
 
CAPTURE_NAME                   RULE_SET_NAME                  PURPOSE
------------------------------ ------------------------------ -------------------
OGG$CAP_GG1_EXT1               OGG$GG1_EXT1_CAPTURE_I         GoldenGate Capture
OGG$CAP_GG1_EXT2                                              GoldenGate Capture

OGG$CAP_GG1_EXT1 was created using GoldenGate 11.2.1.0.3 while OGG$CAP_GG1_EXT2 was created using 11.2.1.0.6BP3. Notice how RULE_SET_NAME is empty for the second one? Without the rule set the Capture process will just capture everything that's happening in the database so my first concern was whether it's going to negatively affect the performance?

Indeed, if we take a look at the stats for both processes...

 
       SID CAPTURE_NAME     STARTUP_TIME          TOTAL_PREFILTER_DISCARDED TOTAL_MESSAGES_ENQUEUED
---------- ---------------- --------------------- ------------------------- -----------------------
        33 OGG$CAP_GG1_EXT1 23/07/2013 2:21:56 PM                        47                      71
       147 OGG$CAP_GG1_EXT2 23/07/2013 2:21:56 PM                        28                    2589

...notice a big difference in TOTAL_MESSAGES_ENQUEUED. Both processes were started at exactly the same time and they are capturing from exactly the same objects. This difference makes sense -- without a rule set in place the second process will enqueue every change it sees. By itself, this change would be pretty bad, but something else has happened as well:

SQL> select capture_name, v.program
  2   from v$streams_capture c, v$session v
  3   where c.SID = v.SID;
 
CAPTURE_NAME                   PROGRAM
------------------------------ ------------------------------------------------
OGG$CAP_GG1_EXT1               oracle@gg1.quadro.com (CP01)
OGG$CAP_GG1_EXT2               extract@gg1.quadro.com (TNS V1-V3)

Notice how OGG$CAP_GG1_EXT1 is an Oracle shadow process (a Streams Capture process) while OGG$CAP_GG1_EXT2 is the Extract process itself! What we had before is the Extract process acting as an XStreams client to the Streams Capture process. With this change it is no longer required and we can see that the XStreams client is now detached from the server:

SQL> select server_name, capture_name, status from dba_xstream_outbound order by 1;
 
SERVER_NAME                    CAPTURE_NAME                   STATUS
------------------------------ ------------------------------ --------
OGG$GG1_EXT1                   OGG$CAP_GG1_EXT1               ATTACHED
OGG$GG1_EXT2                   OGG$CAP_GG1_EXT2               DETACHED

That clears the concern of an empty rule set -- the Extract can now filter the records internally because nothing needs to be shipped out anyways where before it would have resulted in a massive traffic between the Capture process, buffered queue and the XStreams client.