Search

Top 60 Oracle Blogs

Recent comments

Oakies Blog Aggregator

#Exasol on #AWS: Elasticity with #Cloud UI

This is the second part of the mini-series Exasol on AWS. Here’s the first part.

Cloud UI is an extension to EXAoperation that makes it easy for you to

  • Scale up & down
  • Increase storage capacity
  • Scale out by adding nodes to the cluster

Cloud UI can be reached by adding the port number 8835 to the URL of your License Server and uses the same credentials as EXAoperation.

Scale down to m5.large with Cloud UI

Depending on the load you get on your Exasol cluster, you can scale up your data nodes to more powerful EC2 instances if load is high and scale down to less expensive EC2 instances with lower user demands.

I started my little cluster with r5.large instances. Now I want to scale down to m5.large. Enter Cloud UI:

https://uhesse.files.wordpress.com/2019/12/ela01.png?w=1240&h=678 1240w, https://uhesse.files.wordpress.com/2019/12/ela01.png?w=150&h=82 150w, https://uhesse.files.wordpress.com/2019/12/ela01.png?w=300&h=164 300w, https://uhesse.files.wordpress.com/2019/12/ela01.png?w=768&h=419 768w, https://uhesse.files.wordpress.com/2019/12/ela01.png?w=1024&h=559 1024w" sizes="(max-width: 620px) 100vw, 620px" />

You see on the right site that scaling down to m5.large reduces both available memory and costs. I click on APPLY now and confirm the pop-up coming next with EXECUTE. The following steps the system goes through can be monitored in EXAoperation:

https://uhesse.files.wordpress.com/2019/12/ela02.png?w=1240&h=494 1240w, https://uhesse.files.wordpress.com/2019/12/ela02.png?w=150&h=60 150w, https://uhesse.files.wordpress.com/2019/12/ela02.png?w=300&h=119 300w, https://uhesse.files.wordpress.com/2019/12/ela02.png?w=768&h=305 768w, https://uhesse.files.wordpress.com/2019/12/ela02.png?w=1024&h=407 1024w" sizes="(max-width: 620px) 100vw, 620px" />

Notice that the database got restarted during that process.

Scale out by adding data nodes

I want to expand my present 1+0 cluster to a 2+1 cluster. First I add another active node:

https://uhesse.files.wordpress.com/2019/12/ela03.png?w=1240&h=572 1240w, https://uhesse.files.wordpress.com/2019/12/ela03.png?w=150&h=69 150w, https://uhesse.files.wordpress.com/2019/12/ela03.png?w=300&h=138 300w, https://uhesse.files.wordpress.com/2019/12/ela03.png?w=768&h=354 768w, https://uhesse.files.wordpress.com/2019/12/ela03.png?w=1024&h=472 1024w" sizes="(max-width: 620px) 100vw, 620px" />

As you see, this doesn’t only increase the overall avaible memory but also the compute power. Storage capacity is usually also increased when adding a node. In this particular case not, though, because I will also go from redundancy 1 to redundancy 2.

The log looks like this now:

https://uhesse.files.wordpress.com/2019/12/ela04.png?w=1240&h=436 1240w, https://uhesse.files.wordpress.com/2019/12/ela04.png?w=150&h=53 150w, https://uhesse.files.wordpress.com/2019/12/ela04.png?w=300&h=105 300w, https://uhesse.files.wordpress.com/2019/12/ela04.png?w=768&h=270 768w, https://uhesse.files.wordpress.com/2019/12/ela04.png?w=1024&h=359 1024w" sizes="(max-width: 620px) 100vw, 620px" />

My one node cluster did use redundancy 1, now I want to change that to redundancy 2. That step is of course not required when you started with a multi-node cluster using redundancy 2 already. See here for more details about redundancy in Exasol.

To increase redundancy, I go to the EXAstorage page of EXAoperation:

https://uhesse.files.wordpress.com/2019/12/ela05.png?w=116 116w, https://uhesse.files.wordpress.com/2019/12/ela05.png?w=231 231w" sizes="(max-width: 475px) 100vw, 475px" />

The new EC2 instance for the new data node can be renamed like this:

https://uhesse.files.wordpress.com/2019/12/ela06.png?w=150&h=24 150w, https://uhesse.files.wordpress.com/2019/12/ela06.png?w=300&h=48 300w, https://uhesse.files.wordpress.com/2019/12/ela06.png?w=768&h=123 768w, https://uhesse.files.wordpress.com/2019/12/ela06.png?w=1024&h=164 1024w, https://uhesse.files.wordpress.com/2019/12/ela06.png 1049w" sizes="(max-width: 620px) 100vw, 620px" />

That makes it easier to identify the nodes, for example when associating elastic IPs to them. I do that now for n12 in the same way I did it with n11 before.

The elastic IPs of the data nodes must then be entered into the connection details of clients like DbVisualizer in this example:

https://uhesse.files.wordpress.com/2019/12/ela07.png?w=150 150w, https://uhesse.files.wordpress.com/2019/12/ela07.png?w=300 300w" sizes="(max-width: 540px) 100vw, 540px" />

After having added a new active node, that node is initially empty unless REORGANIZE operations are done. For example a REORGANIZE DATABASE:

https://uhesse.files.wordpress.com/2019/12/ela08.png?w=150&h=53 150w, https://uhesse.files.wordpress.com/2019/12/ela08.png?w=300&h=107 300w, https://uhesse.files.wordpress.com/2019/12/ela08.png?w=768&h=273 768w, https://uhesse.files.wordpress.com/2019/12/ela08.png?w=1024&h=364 1024w, https://uhesse.files.wordpress.com/2019/12/ela08.png 1192w" sizes="(max-width: 620px) 100vw, 620px" />

I have a 2+0 cluster now: Mirrored segments on two active nodes but no reserve node.

Adding reserve nodes

To get a 2+1 cluster, I need to add a reserve node. Again, that’s quite easy to do with Cloud UI:

https://uhesse.files.wordpress.com/2019/12/ela09.png?w=1240&h=518 1240w, https://uhesse.files.wordpress.com/2019/12/ela09.png?w=150&h=63 150w, https://uhesse.files.wordpress.com/2019/12/ela09.png?w=300&h=125 300w, https://uhesse.files.wordpress.com/2019/12/ela09.png?w=768&h=321 768w, https://uhesse.files.wordpress.com/2019/12/ela09.png?w=1024&h=427 1024w" sizes="(max-width: 620px) 100vw, 620px" />

Within about 10 Minutes, the log should show something like this:

https://uhesse.files.wordpress.com/2019/12/ela10.png?w=150 150w, https://uhesse.files.wordpress.com/2019/12/ela10.png?w=300 300w" sizes="(max-width: 616px) 100vw, 616px" />

Notice that there was no database restart this time. The new node should get renamed and have a new elastic IP associated as shown before. Also that IP needs to be added to client connection details. See here if you wonder what reserve nodes are good for.

Now that I have got a 2+1 Exasol cluster running on AWS, I’m ready to demonstrate what happens if one node fails. That will be the next part of this series </p />
</p></div>

    	  	<div class=

Securing Data in Power BI- an Interactive Diagram

I haven’t been very good at sharing my analytics work as often I feel others are more skilled in this area than I am, just starting out, but today I’d like to start to fix that by sharing an interactive diagram.  I know, some of you may be saying, “infographic”, but as it’s has links vs. a paginated design, I’ll call it as I see it.

Not the Kitchen Sink

#ffffff; color: #333333; cursor: text; font-family: Georgia,'Times New Roman','Bitstream Charter',Times,serif; font-size: 16px; font-style: normal; font-variant: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration: none; text-indent: 0px; text-transform: none; -webkit-text-stroke-width: 0px; white-space: normal; word-spacing: 0px;">This diagram I built for a customer who was interested in how data was secured at different stages of the Power BI environment and was built with their specific Power BI eco-system in mind.  Due to that. it’s composed of Power BI Pro, the Power BI shared service, on-prem data sources/files, SQL Server Analysis Server, along with Office 365.  This interactive diagram contains links at each stage to Microsoft documentation to gain more in-depth understanding of the feature or data security features.

It does include information about Power BI roles and responsibilities, but doesn’t go into the Azure Active Directly layer, (it’s a busy diagram as it is!)  If the arrows are a bit confusing, also know this customer was using only direct query for their reports/dashboards, as they have very strict governance over data sharing in Power BI.

Downloadable

As the file does have a number of links, the file can be downloaded.  This file has all the links and is a simple Power Point file:

And There’s More

As there are still an abudant number of areas that are of a definitive interest to the security professional, I’ve added a few additional links I think are important to any organization that is facing data governance with Power BI analytics:

I hope this is helpful and if you have any additional areas that you’d like to see covered, click on the comment section for this post and let me know!



Tags:  , , ,


Del.icio.us



Facebook

TweetThis

Digg

StumbleUpon




Copyright ©  [Securing Data in Power BI- an Interactive Diagram], All Right Reserved. 2020.

E-rows / A-rows

This note was prompted by an error I made at the UKOUG TechFest19 yesterday. It’s fairly well-known that when you read an execution plan that includes the rowsource execution stats – so you get the E-rows (estimated) and A-rows (Actual) reported – then a sensible check of the quality of the optimizer’s calculations is to compare the estimates and actuals allowing for the fact that the E-rows is “per start” and the A-rows is “cumulative”, so A-rows = E-rows * Starts.

The error I made yesterday was to forget that this relationship isn’t always true. In particular partitioning and parallel query introduced the need to be a little flexibility in reading the numbers – which I’ll demonstrate with a coupld of simple examples running under 12.2.0.1


rem
rem     Script:         estimate_actual.sql
rem     Author:         Jonathan Lewis
rem     Dated:          Nov 2019
rem

create table pt_composite_1 (
        id,
        grp,
        small_vc,
        padding
)
nologging
partition by range(id) 
subpartition by hash (grp)
subpartitions 4
(
        partition p1 values less than (  4000),
        partition p2 values less than (  8000),
        partition p3 values less than ( 16000),
        partition p4 values less than ( 32000),
        partition p5 values less than ( 64000),
        partition p6 values less than (128000)
)
as
with generator as (
        select 
                rownum id
        from dual 
        connect by 
                level <= 1e4 -- > comment to avoid WordPress format issue
)
select
        rownum                                            id,
        trunc(rownum/100)                                 grp,
        cast(to_char(trunc(rownum/20)) as varchar2(10))   small_vc,
        cast(rpad('x',100) as varchar2(100))              padding
from
        generator       g1,
        generator       g2
where 
        rownum <= 1e5 -- > comment to avoid WordPress format issue
/

create table t3 
nologging pctfree 80
storage (initial 1M next 1M)
as
select * from pt_composite_1
/

All I’ve done is create a couple of tables with 100,000 rows each – and now I’m going to count the rows and see what I get from the execution plans with rowsource execution stats enabled:


set serveroutput off
alter session set statistics_level = all;

prompt  =================
prompt  Partition effects
prompt  =================

select count(id) from pt_composite_1;
select * from table(dbms_xplan.display_cursor(null,null,'allstats last'));

prompt  ================
prompt  Parallel effects
prompt  ================

select /*+ parallel (t3 4) */ count(id) from t3;
select * from table(dbms_xplan.display_cursor(null,null,'allstats'));

With a little cosmetic tidying, here are the two execution plans (note that I haven’t used the “last” format option when reporting the parallel plan:


=================
Partition effects
=================

select count(id) from pt_composite_1

-------------------------------------------------------------------------------------------------
| Id  | Operation            | Name           | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
-------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT     |                |      1 |        |      1 |00:00:00.04 |    1866 |
|   1 |  SORT AGGREGATE      |                |      1 |      1 |      1 |00:00:00.04 |    1866 |
|   2 |   PARTITION RANGE ALL|                |      1 |    100K|    100K|00:00:00.04 |    1866 |
|   3 |    PARTITION HASH ALL|                |      6 |    100K|    100K|00:00:00.04 |    1866 |
|   4 |     TABLE ACCESS FULL| PT_COMPOSITE_1 |     24 |    100K|    100K|00:00:00.04 |    1866 |
-------------------------------------------------------------------------------------------------


================
Parallel effects
================

select /*+ parallel (t3 4) */ count(id) from t3

------------------------------------------------------------------------------------------------------
| Id  | Operation              | Name     | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |
------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT       |          |      1 |        |      1 |00:00:00.04 |      20 |      0 |
|   1 |  SORT AGGREGATE        |          |      1 |      1 |      1 |00:00:00.04 |      20 |      0 |
|   2 |   PX COORDINATOR       |          |      1 |        |      4 |00:00:00.04 |      20 |      0 |
|   3 |    PX SEND QC (RANDOM) | :TQ10000 |      0 |      1 |      0 |00:00:00.01 |       0 |      0 |
|   4 |     SORT AGGREGATE     |          |      4 |      1 |      4 |00:00:00.11 |    8424 |   7692 |
|   5 |      PX BLOCK ITERATOR |          |      4 |    100K|    100K|00:00:00.11 |    8424 |   7692 |
|*  6 |       TABLE ACCESS FULL| T3       |     61 |    100K|    100K|00:00:00.06 |    8424 |   7692 |
------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   6 - access(:Z>=:Z AND :Z<=:Z)

As you can see, the lines specifying partition selection report E-Rows for the whole table, not for any partition-level approximation, so for operations 3 and 4 we shouldn’t multiply Starts by E-rows to compare with A-row. (Starts = 6 for operation 3 because we have 6 partitions, and Start = 24 for operation 4 because at the lowest level we have a total of 24 data segments).

For the parallel query we see the same pattern – every parallel slave reports the expected total number of rows, and every “block iterator” (rowid range) reports the expected total number of rows. Again we see that using multiplication to compare E-rows and A-rows would not be valid.

In fact it’s not just partitioning and parallelism that can cause confusion. Even something as simple as a serial nested loop join has a couple of surprises (largely thanks to the evolution of the mechanics – without a matching adjustment to the execution plans – over time). Here’s a script to generate a couple of tables, which we will then join – hinting various mechanisms for the nested loop.


create table t1
as
select
        rownum           id,
        mod(rownum,100)  n1,
        cast(lpad(rownum,20) as varchar2(20)) v1 
from
        dual
connect by
        level <= 1000 -- > comment to avoid WordPress format issue
;

create table t2
as
select  * from t1
union all
select  * from t1
union all
select  * from t1
;

create index t2_i1 on t2(id);

It’s not a subtle test – as you can see we have 3 rows in table t2 for every row in t1. So let’s pick some t1 rows and join to t2 on id. Again it’s 12.2.0.1:

set serveroutput off
alter session set statistics_level = all;

prompt  ==============================
prompt  Nested loop join (traditional)
prompt  ==============================

select
        /*+ 
                leading(t1 t2) use_nl_with_index(t2 t2_i1) 
                opt_param('_nlj_batching_enabled', 0)
                no_nlj_prefetch(t2)
        */
        t1.v1, t2.v1
from
        t1, t2
where
        t1.n1 = 4
and
        t2.id = t1.id
;

select * from table(dbms_xplan.display_cursor(null,null,'allstats last'));

prompt  ==============================
prompt  Nested loop join with prefetch
prompt  ==============================

select
        /*+ 
                leading(t1 t2) use_nl_with_index(t2 t2_i1) 
                nlj_prefetch(t2) 
        */
        t1.v1, t2.v1
from
        t1, t2
where
        t1.n1 = 4
and
        t2.id = t1.id
;

select * from table(dbms_xplan.display_cursor(null,null,'allstats last'));

prompt  ==============================
prompt  Nested loop join with batching
prompt  ==============================

select
        /*+ 
                leading(t1 t2) use_nl_with_index(t2 t2_i1) 
                nlj_batching(t2) 
        */
        t1.v1, t2.v1
from
        t1, t2
where
        t1.n1 = 4
and
        t2.id = t1.id
;

select * from table(dbms_xplan.display_cursor(null,null,'allstats last'));

Here are the three plans, with a couple of comments after each – all three queries returned the same 30 *- 10 * 3) rows.


==============================
Nested loop join (traditional)
==============================

--------------------------------------------------------------------------------------------------------
| Id  | Operation                            | Name  | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
--------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                     |       |      1 |        |     30 |00:00:00.01 |      60 |
|   1 |  NESTED LOOPS                        |       |      1 |     30 |     30 |00:00:00.01 |      60 |
|*  2 |   TABLE ACCESS FULL                  | T1    |      1 |     10 |     10 |00:00:00.01 |      15 |
|   3 |   TABLE ACCESS BY INDEX ROWID BATCHED| T2    |     10 |      3 |     30 |00:00:00.01 |      45 |
|*  4 |    INDEX RANGE SCAN                  | T2_I1 |     10 |      3 |     30 |00:00:00.01 |      15 |
--------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - filter("T1"."N1"=4)
   4 - access("T2"."ID"="T1"."ID")

This is the original nested loop structured (apart from the “batched” option that appeared in 12c) and follows the rule/guideline:

  • Operation 2 operates once and returns the 10 rows predicted.
  • Operation 3 is started 10 times by operation 1, with a prediction of 3 rows per start – and the actual comes out at 30 rows.
  • Operation 4 is started 10 times (once for each start of operation 3), with a predication of 3 rowids per start – and the actual comes out at 30 rows
==============================
Nested loop join with prefetch
==============================

-------------------------------------------------------------------------------------------------------
| Id  | Operation                           | Name  | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
-------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                    |       |      0 |        |      0 |00:00:00.01 |       0 |
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| T2    |      1 |      3 |     30 |00:00:00.01 |      60 |
|   2 |   NESTED LOOPS                      |       |      1 |     30 |     30 |00:00:00.01 |      30 |
|*  3 |    TABLE ACCESS FULL                | T1    |      1 |     10 |     10 |00:00:00.01 |      15 |
|*  4 |    INDEX RANGE SCAN                 | T2_I1 |     10 |      3 |     30 |00:00:00.01 |      15 |
-------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   3 - filter("T1"."N1"=4)
   4 - access("T2"."ID"="T1"."ID")
 

Again in the order of rowsource generation

  • Operation 3 starts once with a prediction of 10 rows and the rule works.
  • Operation 4 is started 10 times by operation 2, with a prediction of 3 rows (rowids) per start, and the rule works.
  • Operation 2 was started once by operation 1, with a predication of 30 rows (rowids), and the rule works.
  • Operation 1 starts once, but the prediction is reported as the value you would have got from the original NLJ shape – and breaks the rule.
==============================
Nested loop join with batching
==============================

------------------------------------------------------------------------------------------------
| Id  | Operation                    | Name  | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |       |      1 |        |     30 |00:00:00.01 |      60 |
|   1 |  NESTED LOOPS                |       |      1 |     30 |     30 |00:00:00.01 |      60 |
|   2 |   NESTED LOOPS               |       |      1 |     30 |     30 |00:00:00.01 |      30 |
|*  3 |    TABLE ACCESS FULL         | T1    |      1 |     10 |     10 |00:00:00.01 |      15 |
|*  4 |    INDEX RANGE SCAN          | T2_I1 |     10 |      3 |     30 |00:00:00.01 |      15 |
|   5 |   TABLE ACCESS BY INDEX ROWID| T2    |     30 |      3 |     30 |00:00:00.01 |      30 |
------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   3 - filter("T1"."N1"=4)
   4 - access("T2"."ID"="T1"."ID")


In the order in which rowsources are created

  • Operation 3 starts once with a prediction of 10 rows – and the A-rows matches the rule
  • Operation 4 is started 10 times by opreation 2, with a prediction of 3 rows per start – and the A-rows matches the rule.
  • Operation 5 is started 30 times by operation 1, with a prediction of 3 rows per start – again reporting the value that you would have seen from the original representation of the NLJ, the prediction obviously should be 1 – so the rule is broken again

tl;dr

It is important to remember that the basic rule of “A-rows = starts * E-rows” does not hold for the partition-related lines or the PX related lines of partitioned and parallel execution plans.

You may also find a few other cases where you need be a little cautious about trusting the rule without first thinking carefully about the mechanics of what the shape of the plan is telling you.

Oracle 19c Real-Time and High-Frequency Automatic Statistics Collection

I gave this presentation at the UKOUG Techfest 19 conference.  This video was produced as a part of the preparation for that session.  The slide deck is also available on my website.


It takes a look at the pros and cons of these new 19c features.  They are only available on Engineered Systems.  Both features aim to address the challenge of using data that has been significantly updated before the statistics maintenance window has run again.

  • Real-Time Statistics uses table monitoring to augment existing statistics with simple corrections
  • High-Frequency Automatic Optimizer Statistics is an extra statistics maintenance window that runs regularly to update the stalest statistics.

As your statistics change, so there are opportunities for SQL execution plans, and therefore application performance to change. DBAs and developers need to be aware of the implications.

Practical Application Performance Tuning: Applying Theory in Practice

I gave this presentation at the UKOUG Techfest 19 conference.  It is closely based on a previous presentation about PeopleSoft nVision performance tuning, and uses the experience of a PeopleSoft project as a case study, so I am also posting here and on my PeopleSoft blog.
This video was produced as a part of the preparation for this session.  The slide deck is also available on my website.


Learning about and understanding the principles and mechanics of the Oracle database is fundamentally important for both DBAs and developers. It is one of the reasons we still attend physical conferences.
This presentation tells the story of a performance tuning project for the GL reporting on a Financials system on an Oracle engineered system. It required various database techniques and features to be brought to bear. Having a theoretical understanding of how the database and various features work allowed us to make reasonable predictions about whether they would be effective in our environment. Some ideas were discounted, some were taken forward.
We will look at instrumentation, ASH, statistics collection, partitioning, hybrid columnar compression, Bloom filtering, SQL profiles. All of them played a part in the solution, some were simple and effective, some will require periodic maintenance, some added further complications that had to be worked around, some had to be carefully configured in-line with the application, and some required some reconfiguration of the application into order to work properly.

Practical Application Performance Tuning: An nVision Case Study

I gave this presentation at the UKOUG Techfest 19 conference.  It is closely based on a previous presentation about PeopleSoft nVision performance tuning, and uses the experience of a PeopleSoft project as a case study, so I am also posting both here on my PeopleSoft blog, and also on my Oracle blog.
This video was produced as a part of the preparation for this session.  The slide deck is also available on my website.


Learning about and understanding the principles and mechanics of the Oracle database is fundamentally important for both DBAs and developers. It is one of the reasons we still physical conferences.
This presentation tells the story of a performance tuning project for the GL reporting on a Financials system on an engineered system. It required various techniques and features to be brought to bear. Having a theoretical understanding of how the database and various features work allowed us to make reasonable predictions about whether they would be effective in our environment. Some ideas were discounted, some were taken forward.
We will look at instrumentation, ASH, statistics collection, partitioning, hybrid columnar compression, Bloom filtering, SQL profiles. All of them played a part in the solution, some added further complications that had to be worked around, some had to be carefully integrated with the application, and some required some reconfiguration of the application into order to work properly.
Ultimately, performance improvement is an experimental science, and it requires a similar rigorous thought process.

Video : SQLcl and Oracle REST Data Services (ORDS)

In today’s video we’ll demonstrate the ORDS functionality built into Oracle SQLcl.

This is based on this article.

There are loads of other ORDS articles here.

The star of today’s video is Arman Sharma, captured at Sangam 2015. Seems like yesterday.

Cheers

Tim…


Video : SQLcl and Oracle REST Data Services (ORDS) was first posted on December 2, 2019 at 9:41 am.
©2012 "The ORACLE-BASE Blog". Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement.

Friday Philosophy – Computer Magazines & Women (Not) In I.T

I often get into discussions about Women In IT (#WIT), even more so in the last 4 or 5 years with my growing involvement in organising and being at conferences. There is no doubting that the I.T industry is generally blighted by a lack of women and other minorities (and I don’t like referring to women as “minorities” as there are more women in the UK than men). Ours is mostly a white, male, middle-class and (especially in the Oracle sphere) “middle aged” world.

I’ve never been happy with the ratio of men to women in the IT workplace – and I started my career in the UK National Health Service, where the ratio of men to women in technical roles seemed more like 80:20. In all companies since, the ratio I would estimate as been 10-15% women. And I haven’t seen it changing much. And I’m afraid to say, to a certain degree, I have almost given up on trying to correct this imbalance in our current workforce. Note, current workforce.

Why? Well, I’ve tried for years to increase the ratio of women in technical areas or at least to increase female representation. That is, make women more visible:

  • When I’ve hired new staff I’ve given female candidates an extra half point in my head – and part of me hates doing it because it’s sexist, the very thing that is the problem. But the small wrong done to try and right a larger wrong.
  • When allocating pay increases I looked out for imbalance (is Sarah doing the same role as Dave to the same level, but being paid less? Let’s fix that).
  • When I have input to paper selection for conferences, “minorities” get an extra half point. But only half. They have to be good at presenting/have an interesting abstract.
  • When it comes to promotion, it is utterly on merit. I don’t care what’s in your underwear, the colour you are, what clothes you wear that are dictated by religion. If your work is deserving of promotion and I can promote, I promote. No positive or negative discrimination. I take this stance as I know people do not want to be promoted “just because” of filling a quota. Further, if it is perceived that this is happening, it creates a bad backlash.

But, really, it’s had little impact. The problem I keep hitting is that there are simply far fewer women in I.T. We can all try and skew things in the way that I (and many others) do or strive for more women in visible positions to act as role models, which I think is an important thing for our industry to do.

But we can’t magically create more women in I.T. Specifically, we can’t create women who have been doing the job for a long time and so are more likely to be skilled and willing to present. We can only work with what we have. One result of the skewing is a relatively small number of women are constantly asked to present and invariable sit on #WIT panels. We see the same people over and over again.

What we can do is encourage a more mixed group of young people coming into the industry. It won’t help much with something like the database world, or at least the database user community, as you see few young people of any type coming in – we need to fix that as well and I applaud things like the German user group #NextGen efforts – databases do not attract young people, It’s Not Cool. But that’s a whole other topic for another day.

In discussing all this, many times, over the years the idea that we need to go back to pre-work people (that would be kids and teenagers then) and encourage everyone – irrespective of gender,sexuality, ethnicity etc etc etc – to do IT, Science, Art, domestic science, whatever they want and ignore the stereotypes of old – is pretty much agreed to be A Good Thing.

All of this is great but it left me with a question. How did we get into this mess in the first place? Why are there so few women in IT between the ages of 35 and retirement? In the early days a lot of women were in IT compared to the average number of women in scientific areas generally. When I was at school (1980’s) they introduce Computer Studies into the curriculum and there were as many girls as boys in my class. Ability was equally spread. The number of women taking IT at college was admittedly terribly low when I went, but colleges did a lot to encourage women and the numbers were rising. And then stopped. Why? What was stopping girls continuing with computers? Well, a year or two ago I read an article (I think in print as I struggled to find similar online – but if you find one let me know) about the computer press back in the 90’s. And it stuck a chord with me.

https://mwidlake.files.wordpress.com/2019/11/screenhunter_-269.jpg?w=123... 1234w, https://mwidlake.files.wordpress.com/2019/11/screenhunter_-269.jpg?w=150... 150w, https://mwidlake.files.wordpress.com/2019/11/screenhunter_-269.jpg?w=300... 300w, https://mwidlake.files.wordpress.com/2019/11/screenhunter_-269.jpg?w=768... 768w, https://mwidlake.files.wordpress.com/2019/11/screenhunter_-269.jpg?w=102... 1024w" sizes="(max-width: 617px) 100vw, 617px" />

The article argued that part (not all, but maybe a big part) of the problem was the computer magazines of the time. I’ve picked on “PC Format” as it was a magazine I bought often and knew, but others were similar. PC Format seemed to me to nearly always have a sexualised image of a woman on the cover, like the one at the top of this article. This was especially true if the image was a bit “science fiction”, say a ray-traced image to promote graphics cards. The image would invariably be of a woman with a, frankly, quite striking and often physiologically unlikely figure. Inside the magazine adverts were liberally decorated with nubile women leaning forward provocatively or with striking make-up & hair and yet wearing nerd glasses. You know, the sort of look you NEVER saw in real life. This was not a style or fashion magazine, it was not an “adult” magazine, it was about mother boards, CPUs, games, programming and general tech.

The covers I found online for this article are not as bad as many I remember (and perhaps I should not be using the worst anyway), but you get the idea. And it was not just PC Format, but that particular publication seemed to style itself as more a lifestyle magazine than just Tech or just Games. Games magazines also had a fair amount of “Dungeons & Dragons” images of women wearing clothes you would freeze to death in and be totally unsuitable for a bit of sword fighting. Why all the women?

When I read the article about this sexism I remembered a letter that had been published in, probably, PC Format. That and the response utterly summed it up. The letter asked why the magazine kept using sexy images of women on the front of a computer magazine. It wasn’t very Women’s Lib. The answer by the magazine was basically “If we put a sexy picture of a woman on the front it sells more. The more copies we sell the more money we make. We are simply giving you what you want; it’s not our problem, it’s actually yours”.

At the time I liked that letter as it said “you the public are in the wrong” and I rather liked stuff that put two fingers up at the majority and I mentally supported the magazine’s position. Looking back now, what strikes me is the abject shirking of responsibility and blatant putting profit before morality. Which I think is the biggest blight on society. Now I’m angry that the magazine just shrugged it’s shoulders and kept on.

When you added the magazines to the depictions of women in science fiction films & TV, and then once you were in the industry the use of booth babes and that nearly all women in sales & PR looked more like models than average (which still is true today) then the whole message was “women – you can be OK in IT if you are able to look like and act like this”. It’s not very inclusive.

The odd thing is, If you look further back at the old Sinclair User or Commodore User magazines, they had nothing like the same level of sexualised imagery of women on the front – they mostly had screen shots of the games in them or art work based on the games. The sexism grew through the end of the 80’s and into the 90’s I think.

So what is my point? We see less of this stuff these days, isn’t it more historical? Well, I think we need to keep an eye on history as it informs. I think it also explains (partly) the lack of mature women in I.T and that it’s almost impossible to change now. But also, it’s not so much “don’t repeat the mistakes of the past”  but “what mistakes are we currently making that in 20 years will be as obvious as that old mistake”. It’s not avoiding the same mistakes but similar ones.

I’ve been talking to Abigail Giles-Haigh recently about her presenting at our (UKOUG’s) #WIT event at Techfest 2019.  Abi is an expert on Artificial Intelligence and we were chatting about the dangers of training systems on historic data, as they can perpetuate historical bias. Also, any system we train now can bake in current bias. It might not even be conscious bias, it can be a bias due to an absence of training data. Some face recognition systems struggle to recognise people with dark skin tones for example. It’s not beyond reason that if we were training AI systems back in the 90’s as to what makes a computer magazine popular, it might have picked up on not just the sexualised lady images but also other aspects of an overtly male-oriented magazine, such as the type of adverts or the language used. Adjustements in light of the data would be made, sales would have gone up even further, and locked in the white-male bias. Only now it would be AI driving it and would we question the underlying, unconscious biases? I do think it’s a danger.

I think it’s going to continue to be a real struggle to encourage more non-white-male-old people into the industry, especially if we try and change the mature workforce. I’m not going to stop trying but I honestly don’t think we can make much difference to the here-and-now.

But we can work more to remove bias for the in-coming generation. And for that we need role models. From the current generation.

 

Presenting Well – Tell Your Story

<< I Wish All New Presenters Knew This (and it will help you)
<<<< Controlling The Presentation Monster (Preparing to Present)
<<<<<< How to (Not) Present – The Evil Threes

I don’t think the key to a really good presentation is the content, the structure, the delivery method, or even the main message. It’s The Story.

Actually, I’d go as far as to say that there is no one, single key to presenting well – but The Story seems to be at the heart of many of the best presentations I have seen and I think that some of the best presenters I know use The Story.

More and more I strive to present by Telling A Story. It works for me and since I started doing this, I think my presentations have got a lot better.

When you read (or watch) a story, it is about something – a person, an event, how change occurred, overcoming an obstacle. It might be hard to totally define what a story is, but when you read a book and it does not really go anywhere, it’s usually not satisfying and you know it has not really told the story. Some presentations are like that: They have some great content and there is knowledge being passed on but, just as when characters are poorly developed or the plot is disjointed, the presentation feels like it’s made of bits and you come away feeling you can’t join all the dots. With a book lacking a good story you may feel you did not get the whole thing; with a technical presentation you might feel you don’t really understand how you do something – or why.

When people design a talk they usually focus on “what facts do I need to tell, what details must I include”. The aim is to put information in other people’s heads. But facts and code and details are hard to absorb. For many a story helps it all go in more smoothly. You absolutely need the facts and details, but if you start gently, setting the pace – but maybe hinting of things to come or an early nugget of detail maybe  (as you do with story) – then expand the scope and go into the details you stand a better chance of carrying the crowd with you.

If you are now thinking “It’s hard enough to come up with a presentation topic, design the talk and then deliver it, and now you want me to do all that and in the form of a story?!? – that’s going to be so much harder!” well, let me explain why I think it is actually easier.

It’s already a story

First of all, what you want to talk about could be, by it’s very nature, already a story.

If the presentation is about using a software technique or product to solve a business problem – that’s a story about how you did it (or, even better, how you tried to do it and it failed – most people present on successes but presentations on failures are often fantastic!).

If it is about learning about a feature of a language or of the database, your story is something like:

“how do I get going with this, what do I need to learn, the things that went wrong, my overcoming adversity {my ignorance}, and finally reaching the sunny uphills of expertise”.

Flow

A story has a flow. It’s a lot easier to learn a story than a set of facts. Some talks are just facts. In fact {see what I did there} many techniques for remembering lists of things are to make them into a story.

Rather than making it harder to remember, having a story makes it easier to remember your talk and move through it. Each part of the presentation leads to (and reminds you of, up on that scary stage where your brain might burp) the next part. The Story helps remove the fear of forgetting parts of your material, and thus helps Control the Presentation Monster.

For the audience it gives them a progression, a narrative. I find that if a talk does not so much leap from points but more segues into them, it is easier to listen and focus. As I design my talks and add more facts and details, I keep in mind how can I preserve the flow. If I am going to talk about some of the things that can go wrong, putting them all in 4 slides together is easy for me and I have a chunk of “things to avoid” – but it may well break the flow, so I try to mention the things to avoid as I came across them or as I expand my theme. I fit them into the flow of the story.

Added colour

I’m not at all suggesting you invent characters or plot devices for your talk. That really would be hard! I also suspect that, unless you were a brilliant story teller, it would be pretty awful! But you can add in little aspects of this.

If I mention someone in my presentation, I usually give a couple of bits of information about them. Not a biography, just something like “Dave was the systems admin – wonderful collection of Rick & Morty t-shirts and no sense of smell”. There is no need for me to do this, it does not help understand the technical content, but now people have a mental (and possibly even nasal) image of Dave.

Side plots – if in learning about some aspect of say Virtual Private Database I discovered something about PL/SQL functions, I’ll divert from My Core Story and give 3 or 4 minutes on that (as a mini story). The great thing about side stories is that, depending on your time management, you can drop or include them as your talk progresses. If I get asked questions during my talk and it has slowed me down (which is NOT a problem – I love the interaction) I can drop a side plot.

Interaction

Finally, when you tell a story you talk to your audience. You are not talking AT an audience. You are explaining to them the background, taking them through the narrative of the topic and leading them, possibly via some side stories, to the conclusion. It is far more like communicating with your audience than dictating to them. And, if you are brave enough to do so, you can look at your audience and engage with them, try to judge if they are following the story and have any feedback or response to it. Mostly any feedback is quite passive (no one shouts out to hear more about PL/SQL functions) but you will catch people’s eye, get a smile, get some indication that they are listening.

For me, discovering that last bit about The Story was when I finally felt I had a way of presenting that worked for me. If I am talking with my audience and I feel there is an engagement, a rapport, that is when I do my best job of it. That’s when I come off the stage buzzing and happy.

Danger Will Robinson!

There is a danger to Telling a Story and that is time. Most good stories build to a satisfying end. Most technical presentations also generally have a main point. But if you are progressing through a Story you might run out of time, in which case you do not get to your Big Expose or you have to suddenly blurt out the ending. It’s like those TV programs where they obviously run out of steam and some kludge is used to end it  – “And then the side character from an hour ago appears, distracts the dragon and you nick the golden egg! Hurr…ah?”.

You can modify the run time with side plots as I say above, but if you are going to Tell a Story, you need to practice the run time more than normal.

You can finish early, it’s better than not finishing at all. But being on time is best.

 

Oracle Cloud : Free Tier and Article Updates

https://oracle-base.com/blog/wp-content/uploads/2019/11/free-220x300.png 220w" sizes="(max-width: 189px) 85vw, 189px" />

Oracle Cloud Free Tier was announced a couple of months ago at Oracle OpenWorld 2019. It was mentioned in one of my posts at the time (here). So what do you get for your zero dollars?

  • 2 Autonomous Databases : Autonomous Transaction Processing (ATP) and/or Autonomous Data Warehouse (ADW). Each have 1 OCPU and 20G user data storage.
  • 2 virtual machines with 1/8 OCPU and 1 GB memory each.
  • Storage : 2 Block Volumes, 100 GB total. 10 GB Object Storage. 10 GB Archive Storage.
  • Load Balancer : 1 instance, 10 Mbps bandwidth.
  • Some other stuff…

I’ve been using Oracle Cloud for a few years now. Looking back at my articles, the first was written over 4 years ago. Since then I’ve written more as new stuff has come out, including the release of OCI, and the Autonomous Database (ADW and ATP) services. As a result of my history, it was a little hard to get exited about the free tier. Don’t get me wrong, I think it’s a great idea. Part of the battle with any service is to get people using it. Once people get used to it, they can start to see opportunities and it sells itself. The issue for me was I already had access to the Oracle Cloud, so the free tier didn’t bring anything new to the table *for me*. Of course, it’s opened the door for a bunch of other people.

More recently I’ve received a few messages from people using the free tier who have followed my articles to set things up, and I’ve found myself cringing somewhat, as aspects of the articles were very out of date. They still gave you then general flow, but the screen shots were old. The interface has come a long way, which is great, but as a content creator it’s annoying that every three months things get tweaked and your posts are out of date. </p />
</p></div>

    	  	<div class=