Global Pragmatica LLC®

Previously I wrote a response to Adriel Hampton’s thought-provoking blog post entitled “Five Reasons to Kill ‘The Meeting'” in which I argued why I think live meetings, preferably in person, are valuable, even though many of us hate a lot of them. Now I’m going to share some tips on how to make your meetings better.

I’m writing this primarily for people who run meetings, but most of these ideas can be used to good effect by mere “powerless” attendees. These are all classic facilitation concepts, and while a designated, trained facilitator will have advantages that attendees don’t, attendees can often speak up and “facilitate from their chair” with astonishing effectiveness, and in some groups, a peer will be far more effective than any authority figure.

What people hate most about meetings is feeling powerless.

Or ignored.

But it’s usually the same thing.

• We all hate going to meetings where we’re talked at and nobody notices or cares if we fall asleep.
• We all hate meetings where the decision has already been made, but nobody’s being up-front about that.
• We all hate meetings where the people who need to hear the discussion aren’t in the room, or aren’t listening, or just don’t get it.
• We all hate meetings where we know we’re going to have the same old fights and end up in the same old impasse, and nobody’s going to make a decision (or realize that their plan hasn’t been working and isn’t likely to).
• We all hate meetings where only one point of view is important. I don’t really care if the CEO thinks this is the only way to save the company, if I know it can’t be done in the time and budget allowed, or if I know that the customers hate it when we do that, or if I know that’s the right thing to do but key stakeholders are too proud to accept a change in plans, or, or, or, or…

Meetings slow things down, and that’s good. (Sometimes.)

Central to many arguments about meetings is a premise that meetings slow things down. Certainly it’s true that many meetings are a waste of time for at least some if not all of the participants, and it’s not uncommon for people to have so many regularly-scheduled meetings that they effectively have only a one- or two-day work week. (More on that below.)

However, I question the premise that speeding things up is a good thing. The more important an outcome is, the more important I think it is to slow down and make sure it’s the right outcome.

“Go slow to go fast” is a facilitator’s mantra. It is far better to waste an hour in a meeting than to proceed with a plan that misses an important detail or a team that isn’t in full agreement.

A single team member who disagrees with the plan can sabotage an entire project. A good facilitator discovers who that person is and makes sure that person has a chance to voice their concerns. A good facilitator helps that person get the chance to explain what the others might not be considering.

Sometimes that person is just a nuisance. But even the troublemakers usually have useful points to make, even if you don’t like the way that they make their points.

When this person’s concerns are heard respectfully, and restated by others so that the person can be confident s/he was understood, then the group can weigh those concerns against the known constraints and competing concerns in a way that either incorporates those concerns or at least enables the person to go along with the plan. Even if the group reaches a different decision, if the dissenting concerns are acknowledged and weighed in a process that is transparent and is consistent with the group’s agreed-upon decision-making method, usually the dissentor(s) will be able to commit to the plan.

More on both of those ideas!

Transparent doesn’t mean public.

When I say that a group (or leader) needs to have a transparent process, that doesn’t necessarily mean that everybody is in on everything. It only means being clear and honest about how information will be explored and how decisions will be made. For example, a leader can say, “I want to get your feedback and ask for a show of hands on the different options today, and I will take that input to next week’s meeting with the directors, who will make a decision.” Most teams will accept that happily, but if they think they get to make the decision and then someone else does, they’ll be angry.

Transparency also requires following through on the stated process and being candid about any subsequent change of course.

Transparency and accountability means not pointing fingers at the team who tried to talk you out of it if it eventually turns out you were wrong. It might kill you to say it, but acknowledging that the team was right and you were wrong will buy you tremendous team loyalty—so much that I’d almost recommend doing that on purpose once. Almost!

Agree on (or at least announce) a decision-making method.

Decisions don’t have to be unanimous or even consensus or majority-rule. Many decision-making methods can work. The most important thing is to have one, and the next most important thing is to have group agreement or at least a candid announcement about what it is.

How do you decide how to decide? It depends on what’s at stake. Generally, the more say a team has in the decisions that affect them, and the more confident the team is that everyone on the team accepts the decisions, the more conscientious that team will be about executing on the decisions and being proactive about resolving issues that arise. The catch is that more say takes longer.

Here are some valid decision-making methods, from fastest and least engaging to slowest and most engaging:

1. Leader decides and announces.
2. Leader seeks input, then decides.
3. Majority rule (discuss and vote).
4. Consensus (keep at it until most people agree and those who disagree are satisfied that their concerns have been addressed or at least acknowledged).
5. Unanimous (keep at it until everybody can agree with the plan).

Having a fallback is helpful. For example, “We want to reach consensus, but if we cannot reach consensus by the end of the week, then we’ll take a vote on Monday.” Or, “If the team can reach a unanimous agreement, that will be the plan, but otherwise I’ll make a decision based on our discussion today.”

When is something important enough to justify a meeting?

What is the value of a good decision, or an effective plan, or a group that agrees enough with the plan to remain committed to it? What is the cost of not reaching these? What is the risk of proceeding without certainty that everyone is onboard with the plan? That is the value of the meeting. The cost of the meeting is the number of people in the room, times the number of hours, times the hourly wage, plus any other costs such as travel, room rental, web-meeting fees, etc. You might multiply the number of hours by the number of people by an average cost of $50 per staff member or manager and $100 per executive or hired consultant. If the value is higher than the cost, you should have a meeting.

It’s often hard to estimate value objectively, but here are some subjective criteria that are probably good enough. Ask yourself these questions about the outcome:

• Will more than a few people spend more than a few weeks working on it?
• Will a customer ever see it?
• Could a bad result lead to a lawsuit?
• Is there anyone affected by it who might be silently disagreeing?
• Is anyone’s influence out of proportion to his or her competence and credibility? (For example, a CEO who doesn’t understand crucial technical details, or a chief engineer who doesn’t understand business constraints, or a sales manager who is purely commission-driven?)
• Are you worried about what you don’t know, or what you might not realize you need to know?

If your answers to any of these questions is yes, then it’s worthwhile to have a meeting.

Minimize the intrusion of meetings on the work week.

Meetings burn time, and not just the duration of the meeting but also the time it takes to get to and from the meeting and time spent with meeting logistics like calendar management, preparation, follow-up, and rescheduling other commitments. Worse, meetings have an interruption cost. If my work requires focused concentration for several hours at a time, then a meeting that runs from 10 to 11 am pretty much destroys my 9 am to lunchtime shift. The most I’ll be able to get done from 9 to 10 and 11 to 12 is handle some email and maybe an expense report or travel reservation. There is no way I’ll be able to debug and fix some code, or write a proposal, or intervene in a staff problem, or persuade my manager about something. If I have another meeting from 2 to 3, then my afternoon is also shot, and my most important responsibilities—the things I’m paid to do—will be postponed another day, or I’ll be forced to put in some overtime that night.

Minimize how your meetings intrude on the work week. Some easy ways to start:

Have designated meeting days.

If you can get all of a team’s meetings out of the way on Tuesdays, that leaves the rest of the week free for focused work. Mondays and Fridays tend to suffer from higher numbers of absences because of people taking long weekends, so Tuesday, Wednesday, and Thursday are better. Ask yourself whether your team benefits from having a break from work mid-week (making Wednesday a good day to meet) or from having several days in a row available to focus (making Tuesday or Thursday better). More important than which day(s) you choose, though, is that you choose, and you enforce the no-meetings-days goal as much as possible.

Have the shortest effective meeting.

Go slow to go fast—make sure everybody’s voice is heard when it’s important—but don’t waste time with unnecessary agenda items. Don’t hesitate to adjourn early. Offer breaks when people get restless, start yawning, or are clearly needing to check their messages. Start on time no matter who’s late, and never go over the allotted time without group agreement and permission for some people to leave. Don’t go overtime if it would deprive people who need to leave of their chance to be heard.

State the agenda and goals in advance.

Nothing is more frustrating than having to go to a meeting whose purpose is unclear, or worse, where some people might have (or seem to have) a hidden agenda. Send out a written agenda, with time allotments for each topic if possible, and with clearly-stated goals. The goals should be measurable and believable, and they should be nouns—for example, “a list of issues needing further exploration” or “a decision about how to proceed” or “a preliminary schedule with major milestones and goal dates.” Ask yourself how the group will know if they met the goal(s) of the meeting.

At the beginning of the meeting, check with the room: “Are these the right goals? Can we do this today? Am I missing something important?”

At the end of the meeting, even if you think you know the answer, ask the room questions like, “Did we meet this goal? Do we need another meeting? Is there anything for next week’s agenda?”

Protect your team from the risks of vague agenda items.

Your agenda might be vague or contain a vague element. If so, take steps to promote confidence that those vague areas will be handled efficiently and nobody will be ambushed by surprises.

For example, if you need to go around the room to get status reports, have everybody remain standing so that nobody will drone on and on. Add a fun element, such as “What progress did you make, is there anything you’re stuck on, and what’s the next movie you want to see?” (If you do something like this, include the list of movies in your meeting notes.)

Sometimes issues arise that might feel like an ambush to some people in the room. Do what you can to make people comfortable raising those hot-button issues, because sweeping them under the rug is never better, but take steps to protect people from unpleasant surprises becoming nightmare scenarios. For example, you might ask, “Does anybody need some time to research this before we discuss what to do next? Is there anybody else that we’ll need to include in this discussion?” Often the best course will be to allow time right away for the basics to be laid out, let people ask any immediate questions, and then schedule further discussion after people have had some time to ponder and research.

If the circumstances demand an immediate decision, do your best to let people be heard, to record objections and unsettled questions, and then take responsibility for the way you proceed. If you must make an executive decision, be transparent about that. Be honest that you’re making a judgment call with incomplete information, and remain accountable for it in future. Do what you can to revisit the unsettled points when time allows. If possible, plan ways to revise the decision as better information becomes available. If your decision turns out badly, be candid about that, too, and acknowledge that some people did raise pertinent objections.

Follow up with brief meeting notes.

Brief is the key here. All you really need is a record of the decisions and agreements, a list of points needing followup, an acknowledgment of any important disagreements or what have you, and a list of open action items with names and goal dates. Some action items might be incomplete, with names, dates, or other details to be determined. If you are ready to include the next meeting’s agenda and logistical details, great.

Always provide a method for people to correct your mistakes and omissions. For example, “Please REPLY ALL with errata and addenda. Have I missed anything important?”

Avoid detailed summaries of who discussed what or disagreed why; you can only lose at this game. Just record what was agreed and decided, and if appropriate also record the points that were left unaddressed, or the objections that were raised, or the points needing further discussion, without commentary. Ask yourself whether anybody who was in the room will be surprised by your notes or would state anything differently. Ask yourself whether somebody who missed the meeting will learn what they need to know.

Sometimes it’s helpful to consult with the room about what should go in the notes, as a way of preventing misunderstanding later on, or even as a way to bring discussion back into focus. For example, after a lengthy discussion or an uneasy resolution, you might ask questions like, “How should I capture this for the notes? Can somebody restate that for me? Does anybody disagree with this proposal? Are there any action items to go with that?”

Overtime costs a lot more than time-and-a-half.

Be especially careful about scheduling meetings that will force people into working overtime. Even if it doesn’t bring a direct labor cost increase, it usually brings a psychological cost increase.

Speaking for myself, I don’t think twice about working overtime to make up for my own poor decisions, for example, or to solve a problem that I just can’t wrap my brain around during the day. But I resent being forced to work overtime because somebody else wasted my time or made a poor decision. If I have concert tickets or a family obligation or am not feeling well, I resent it even more.

I will forgive my colleagues and take one for the team occasionally, and I’ll gladly go the extra mile when it’s the difference between success and failure for something I believe in. (And now that I’m a self-employed consultant who bills by the hour, I am extremely flexible about when those hours need to happen.) But if any work situation (or a social situation, for that matter) creates a pattern of abusing my time, sooner or later I will resent it. And that resentment will cost the organization in terms of my reduced commitment, my less-than-stellar attitude, my frayed nerves, my depressed health, and eventually perhaps even my departure. I won’t sabotage a project—I’m just not wired that way— but you’d better believe it that if you push some people far enough, they will sabotage your project. Maybe not consciously, maybe not deliberately, but they will find ways to undermine even their own success to get back at someone who has done them wrong.

Do you feel the same way? Do your colleagues?

I have some beliefs because of my experiences. You have had different experiences and reached different conclusions. I would love to hear from you!

What am I missing?

What have I gotten wrong?

What do you see differently?

What did I say that surprised you? Do you think I might be right, at least for some people or situations?

What do you think would surprise me? Can you tell me a story from your experiences that would help me understand your point?

Adriel Hampton wrote an interesting blog post entitled “Five Reasons to Kill ‘The Meeting'” that I felt compelled to rebut in his comments thread, and I thought I would write a bit more here on why I believe meetings can be more valuable—and less horrible—than Mr. Hampton and other victims of bad meetings believe, and in my next post, some tips for making that happen.

This post is an elaboration on the comments I made to his post.

Mr. Hampton’s argument is that groups can be more productive by replacing live meetings with online meeting spaces. His reasons are valid, and for many situations I agree with him. Certain kinds of topics can be explored and debated much more effectively in this way. I, too, have a lot of experience with virtual collaboration (in my case, internationally-distributed teams) and have even found that in many ways I can work more productively from a remote office than a common office.

However, even for topics that lend themselves well to offline discussion (such as a team wiki or shared documents), for anything that is important, I have found that final decisions are best made in a live meeting—preferably face to face, if that’s practical, but at least a teleconference. While anything is better than nothing, the effectiveness of the group’s interaction degrades along with the resolution of the meeting method. The most engaging way to meet is to have everybody together in a room—preferably with some social lubricants like snacks, games, a celebratory meal. This is far “stickier” than everybody scattered around the globe, wearing headsets, and distracted by who knows what else. Ask yourself how many times you’ve surfed the web or answered email during phone or web meetings, and then figure that a good portion of your team is at least as distracted as you have been.

Not everybody learns and communicates the same way.

The main point I made in response to Mr. Hampton’s post was that while his suggestions can work very well for visual/verbal learners and communicators, not everybody learns and communicates best by written word and graphics. Many people—in my experience, more people—learn and communicate best by auditory and kinesthetic means.

Here’s the comment I left on his article:

I agree that most work groups hold too many useless meetings, and for people with a visual learning/communicating style, your suggestions will be helpful for many goals that are not being met well by meetings.

The problem is that other people have auditory or kinesthetic learning styles, and they don’t grasp or convey information as comfortably through the written word as you might. Learning and communication styles also break down along another axis, logical vs. social vs. solitary (clearly your style).

If all of your stakeholders are visual, solitary learners, then shared written methods like you’ve described will work very well for a lot of meeting areas. But most workgroups have a mix of learning styles, and in my experience, auditory/social learners are the majority. Your strategies will tend to minimize their contribution.

Another problem is that even among visual, solitary learners, many important topics are best explored with real-time back and forth in which all participants listen as carefully as they talk, seeking to understand as well as to be understood, with clearly understood goals and decision-making methods. If that doesn’t describe the meetings you’re used to attending, I’m not surprised, and no wonder you feel this way! Most of us have attended far more terrible meetings than good ones—myself included.

Most groups benefit from some guidance and ideally instruction from a skilled facilitator. I have experienced for myself many times the incredible difference that good leadership can make, and if the meeting is about something important, hiring an impartial professional facilitator is something you can’t afford not to do. I greatly improved my own effectiveness as a program manager by learning and adapting facilitation principles and techniques, and I went from being someone who dreaded even my own meetings to someone who eagerly looks forward to facilitating for other groups.

Let’s break these ideas down a bit. First, about visual/solitary learners (writers and readers) vs. those other people (talkers, drawers, builders, tryer-outers).

Have you ever had an experience like this?

If you’re reading this article, then we probably have a lot in common. You write. You read. You think. Alone. And you’re good at it. Me, too.

But here’s what happens—right?

You carefully write up a proposal and send it around by email. You take pains to write a thorough discussion, detailed enough but not too long, with supporting illustrations and even good summary bullet points. You put a clear question or call to action at the end. You leave it in Drafts overnight and come back the next morning to fix up a few details before sending it out.

And then nothing happens.

You send another email. No response, or just a few one-liners come back. You phone a key stakeholder or ask them about it when you run into them at the coffee machine, and they say, “Oh, right. I read that, but…” and then they ask questions or raise objections that make it obvious they didn’t understand a thing. You’re pretty sure they didn’t even read it.

It’s frustrating! You know it was all there in your beautifully-written email, and you know that you covered all the most important points. But they don’t get it!

Why not?!

Try not to jump to conclusions. You’ll never know for sure.

• Some people weren’t paying attention.
• Some people read and understood but forgot.
• Some people got behind on their email and are afraid to admit it.
• Some people disagree so violently they can’t even think about your points.
• Some people are too busy.

Story time!

I once had a boss who told me, “If you can’t get it down to one inch, keep it in Drafts until you can.”

Oh, my G-d.

I wanted to strangle her!

But eventually I learned. I found that the shorter my email, the better my chances that she’d sign off and support me later, or answer my question. The longer my email, the more likely I’d get a brusque response that made no sense, or no response at all.

At first I thought she just didn’t appreciate my attention to detail and the subtle nuances of the  situation. Eventually I realized that she appreciated all that and trusted my judgment but didn’t have time to get bogged down in all the grey areas. That was my job, and as long as I kept her informed, she’d support me to the end.

I (eventually) figured out that the thing to do was get her on the phone and tell her I had a plan but that I wanted her ideas on this or that aspect of my plan. She was great at brainstorming solutions and seeing when my thought-framework was off.

She learned, too. She figured out that where she was good at plotting strategy, I was good at anticipating risks. Where she was good at selling ideas, I was good at making sure her plans were bullet-proof. And together, we were better at collaborating over the phone or over lunch, even though sometimes I needed to write an email to myself to figure out what I thought, and sometimes she needed to enjoy a cocktail and ignore my babble while I worked something through.

So what do we writers do about all that?

No matter how well you write, you have to face the fact some people just don’t take in written information. Some people need to:

• talk things out
• touch things
• draw pictures together
• make physical models
• conduct experiments
• listen to descriptions
• see people’s faces
• think “out loud” and ask “dumb” questions
• spell out the details of who, when, what, how

If you’re a good writer and you like working on things alone, in your own time, you might find this frustrating—I sure do!—but remember, other people find it frustrating having to read and work alone.

You’ll come out ahead if you take a variety of approaches.

I wrote more on the topic of written vs. phone and other communication methods in a Point/Counterpoint column with Tina Wuelfing Cargile.

Embrace diversity!

Rather than dwelling on your frustrations, take advantage of people’s differing skills and preferences.

• The people who prefer talking things out are also often good at enrolling others in the decision and will enjoy presenting the plan to other groups (whereas many excellent writers would rather have a root canal than give a presentation).
• The people who like to draw diagrams together often bring new insights because of their superior spacial reasoning abilities.
• The people who like to build prototypes or conduct experiments will help you find the gaps in your plan, and often they’ll come up with improvements on your idea that you’ll wish you thought of. (Ask me how I know.)
• The people who just don’t pay attention to their email are likely to pay closer attention and ask good questions when you talk to them.

But not just any meeting! A good meeting!

And how to have good meetings instead of crappy meetings will be the subject of my next post: How to hold better meetings.

What do you think?

What have I missed?

Which meetings should be killed and which should be resuscitated?

What are the tools you’ve found that work best at replacing meetings?

When I wrote the first JMP Scripting Guide, way back in the dark old days of JMP 4, easily the most difficult chapter to write was the one about data tables, and the most difficult section of that chapter was the one about Row States. I must have spent the better part of a month trying to figure out all the relevant concepts. Not only was JSL still in its gestational period, but I was new to JMP entirely, so I wasn’t just confused about how to work with row states in JSL. I was confused about row states, period.

Slowly I untangled it all, but I had to keep referring to two cheat sheets I’d built for myself, so I decided they’d probably better become a part of the book. These live on in current editions of the JMP Scripting Guide as Table 5.3, “Row states and how they affect JMP’s analyses, charts, and plots,” and Table 5.4, “Operators for converting between numbers and row states.” Then I had to struggle some more to figure out how to use the operators, and on and on. By the time I was done, I still had only a tenuous grip on how to get anything done with row states, but at least I’d gotten to the point where I knew which of my pages to reread and could usually adapt one of my own examples after a few minutes of squinting and swearing.

Fast forward about a dozen years, and I’m programming in JSL full-time for a variety of clients—cool projects for big companies you’ve heard of—but I still cringe when I have to use row state operators. Unless I can refer to another recent project and steal code from myself, I know that I’ve got a good half hour in that darned book and its ridiculously confusing section on row states. I should point out that I haven’t been involved in that book since version 4—since then, it’s been updated by Lee Creighton, Ph.D. (version 5) and Melanie Drake (since version 6). Honestly, I’m not sure whether the section on row states has changed at all, so the fact that I still hate that section and how hard it is for me to read and understand could still be entirely my own fault. I hate that!

In version 8 markers and several other row state operators got a lot more complicated, because JMP 8 introduced marker themes and unicode markers, among several other improvements by the talented developer Xan Gregg, the guy who brought us Graph Builder. The new features are great, but the already confusing row state operators got even more confusing.

So, when Glenn Donahey at W.L. Gore & Associates posed a question about markers on the LinkedIn group “JMP Professional Network,” I thought, “Great! I’ll answer his question and refresh my own memory while I’m at it. Here’s his question:

Simple JSL question – assigning custom markers. This is a simple question, but I can’t seem to find the answer. Starting with a populated data table, one can right click on the row, choose Markers >> Custom… and assign a marker such as “D”. What is the analagous command in JSL?

Unfortunately, my first few attempts to bang out the command he needed didn’t work, and then I got busy with something else. Monday morning I made another attempt at it but once again got myself confused, and then the ever-helpful Mark Bailey, Ph.D., popped in with some answers. Meanwhile, I banged my head against some of the less-documented details of the Marker State() operator, and here’s what I figured out.

Either I’m dumber than the average bear…

This could be the problem. It really could.

…or row state operators are really confusing!

You know what, they are. Regardless of the first part—whether I’m some special kind of stupid or not—row state operators are confusing. And now here are the things I learned (again!) that might be helpful to everyone else!

To set one row’s state at a time, use a pinch of Whatever State( rowNumber )

For example, to set row 1 to pink, which is for no particular reason color #11:

Row State( 1 ) = Color State( 11 );

Or to set row 1 to an inverted solid triangle, which is for no particular reason marker #21:

Row State( 1 ) = Marker State( 21 );

To use Unicode characters as markers, prepare to sweat

Up until JMP 8, there were a few dozen markers known mysteriously as 1 through 31. These are pictured at right and ought to be familiar to veteran JMP users. JMP 8 introduced a way to exploit a whole mess of Unicode characters as markers—not the entire Unicode character set, unfortunately, but the first 19,560 of them—by using 16-bit numbers as arguments. You have three choices for how to give the argument:

1. As an integer from 33 to 19,560, inclusive
2. As a hex number in a string, inside a Hex to Number() operator
3. As a hex number in a four-digit string escaped with \!u

That makes perfect sense, right? No, me either. Let’s go through these one at a time.

1. Integers

Integers are the easy part. We’ve already seen that 1-31 give JMP’s good old-fashioned marker symbols as seen at right. Number 32 is empty. Not coincidentally, the first 32 positions of the Unicode character set are all empty; JMP’s standard markers fill that gap, and #32 is just empty. After that, you start getting Unicode characters by their positions given in any Unicode character table.

Note: be sure to set your Marker Font() preference to a unicode font:

// set the Marker Font to be Arial Unicode MS, Lucida Grande,
// or another font containing an extensive set of Unicode characters
If( Host is( Windows ),
  Preferences( Fonts( English( Marker Font( "Arial Unicode MS", 10 ) ) ) ),
  Preferences( Fonts( English( Marker Font( "Lucida Grande", 10 ) ) ) )
);

Whoa! What are the Unicode characters, anyway?

Windows users have a decent tool for browsing Unicode. Go to Start / Programs / Accessories / System Tools / Character Map. For Font, choose a unicode font such as Arial Unicode MS. To get details for a specific character, click it and look in the lower part of the window:

The Advanced view is marginally more helpful. Searching online for “Unicode character map” yields plenty of useful options.

Mac users have a great built-in tool for browsing Unicode: in System Preferences, choose Language & Text, click the Input Sources panel, and check the Keyboard & Character Viewer from the list of input methods, and check the “Show Input menu in menu bar.” Here’s how that looks in OS 10.6 (Snow Leopard): Now you have an input menu in the upper right corner of the menu bar. From this menu, choose Show Character Viewer, and you get this handy browser of the entire Unicode character set. (The characters shown are limited to those available in fonts installed on your machine. Mac OS X’s best font for this kind of thing is Lucida Grande.)

So, once you find a symbol you like, these tools tell you which character is, but they tell you in hexadecimal notation, e.g. the FOR ALL character above is in position 2200 hexadecimal (base 16). Now, just figure out what that is in decimal (base ten) and you’re all set! To do that, you can either grab your favorite programmer’s calculator, or you can use JMP’s own conversion operator to learn that it’s 8704:

show(hex to number("2200"));

Hex To Number("2200"):8704

Armed with this knowledge, you can set FOR ALL as the marker symbol for row 5 this way:

Row State( 5 ) = Marker State( 8704 );

2. Hex numbers

But it’s kind of silly to work that hard at it. If you’re using Unicode character tables to find symbols, they’re telling you hex numbers, and you might as well just let JMP do the work:

Row State( 5 ) = Marker State( hex to number("2200" );

3. Escaped numbers

This too is making it too hard, though. Why not just use the common \u-escaped notation, e.g. \u2200 to mean “Unicode character in position hex 2200”? In this case, our argument needs to be a string, and the usual \u escape needs to be escaped with both a backslash and a bang, like most other special characters for JSL. So what in the rest of the world is “\u2200” becomes “\!u2200” for JSL, for a command like this:

Row State( 5 ) = Marker State ( "\!u2200" );

This isn’t too bad, but if you want to store a bunch of these in a character data table column, you need to do more escaping to get them to play nicely with JSL.

New Column( "unicode",
  Character, Nominal,
  Formula( "\!"\!\\!!u" || Substr( Hex( Row(), "integer" ), 5, 4 ) || "\!"" )
);

What a mess! Let’s break it down. In the middle we ask for row numbers to be converted from integers to hex, but we only take the last four digits by using Substr().

Around the whole string, we need quotation marks, but we can’t just use them directly—the character column gets string values, which are quoted strings by definition, and those surrounding quotes are thrown away when we read the values out of the column. So we need escaped quotation marks, \!”, before and after. So far we’ve got, for example on row 8704, ” \!”2200\!” “. But we also need \!u before the number, which is two more special characters that need escaping in JSL: backslash (\) and bang (!). To get the backslash, we need to use \!\, and to get the bang, we need \!!. The u is the easy part. So add \!\ and \!! and u before the 2200, and you’ve got ” \!”\!\\!!u2200\!” ”

But it gets worse! Now if you want to ask for markers from those values, you have to parse all that to get it back to “\!u2200”!

I know: “Yikes!”

To put it all together into a useful script, let’s create a data table, give it the 19560 rows we’ll need to see all the supported Unicode characters, and then set marker states For Each Row(). We’ll add a While() in the middle to make sure the data table finishes getting built before JMP tries to run the For Each Row(); if we don’t, we’ll get an error about arguments.

// a data table to see the entire set of standard markers
// and Unicode characters available for markers
New Table( "unicode characters",
  add rows( 19561 ),
  New Property( "plot my markers",
    Overlay Plot(
      X( :X ),
      Y( :Y ),
      Separate Axes( 1 ),
      SendToReport( Dispatch( {}, "Overlay Plot",
      FrameBox, {Frame Size( 1600, 1600 )} ) )
    )
  ),
  New Column( "unicode",
    Character,
    Nominal,
    Formula( "\!"\!\\!!u" || Substr( Hex( Row(), "integer" ), 5, 4 ) || "\!"" )
  ),
  New Column( "X", Numeric, Continuous, Format( "Best", 10 ),
    Formula( Sequence( 1, 140, 1, 1 ) ) ),
  New Column( "Y", Numeric, Continuous, Format( "Best", 10 ),
    Formula( Sequence( 1, 140, 1, 140 ) ) )
);

// use While() to see when the data table is finished -
// else next line processes before the data table is built
// and you get error: "Argument should be row State{1}"
while(is missing(:Y[19561]), wait(.0001) );

// see all markers using unicode arguments
For Each Row( Row State( Row() ) = Marker State( Parse( :unicode ) ) );

// the equivalent command, using integer arguments
For Each Row( Row State( Row() ) = Marker State( Row() ) );

To set several aspect of one row’s state, add a teaspoon of Combined States()

For example, to set row 1 to a solid pink inverted triangle, stick both of the previous examples together with a Combined States() operator. This would be handy for a World War II historian plotting deaths in the Holocaust.

Row State( 1 ) = Combine States( Marker State( 21 ), Color State( 11 ) );

Or to set row 1 to a yellow (color #41) star of David (marker #10017), to plot the most horrendous statistics from the Holocaust, you could set:

Row State( 1 ) = Combine States( Marker State( 10017 ), Color State( 41 ) );

Some interesting Unicode characters to use as markers

Just scroll through the data table created above to see some Unicode characters that might be particularly useful for JMP graph markers, such as:

• Greek and Coptic start at \u0370
• Currency symbols start at \u20A0
• Hebrew, Arabic, etc. start at \u0590
• Fractions and roman numerals start at \u2150
• Box elements and other geometric shapes and dingbats start at \u2500
• Arrows, math operators, and technical symbols start at \u2190
• Weather, pointing hand, smiley faces, gender, and astrology symbols start at \u2600
• Chess pieces, card suits, musical symbols, recycling symbols, dice, monogram, digram, and trigram symbols start at \u2654
• Braille starts at \u2800
• Asian language characters start around \u2E80

Download the script

Don’t bother with copying and pasteing from the examples above—you can download a tidy sample script here:

• [download id=”4″]

What did I forget?

What is still confusing about marker states? Post your questions and comments below, and I’ll do my best to address them in future posts (or in the comments, if they’re not too complex).

Statisticians and data analysts are in a kerfuffle about the recent remarks of AnnMaria De Mars, Ph.D. (President of The Julia Group and a SAS Global Forum attendee) in her blog that the open source statistical analysis tool R is an “epic fail,” or to put it in Twitterese, #epicfail:

I know that R is free and I am actually a Unix fan and think Open Source software is a great idea. However, for me personally and for most users, both individual and organizational, the much greater cost of software is the time it takes to install it, maintain it, learn it and document it. On that, R is an epic fail.

And oh, how the hashtags and comments and teeth-gnashing began!

Nathan Yau’s excellent FlowingData blog recaps the kerfuffle nicely, and his post has accumulated a thoughtful comments thread, as has Dr. De Mars’, to both of which I added my thoughts, expanded here:

To make my prejudices clear, I’ve spent several decades in commercial statistical software development (working in a variety of R&D roles at SYSTAT, StatView, JMP, SAS, and Predictum, and I now do custom JMP scripting, etc., for Global Pragmatica LLC.

I can say with hard-won authority that:

– good statistical software development is difficult and expensive
– good quality assurance is more difficult and expensive
– designing a good graphical user interface is difficult, and expensive
– a good GUI is worthwhile, because the easier it is to try more things, the more things you will try, &
– creative insight is worth a lot more than programming skill

Even commercial software tends to be under-supported, and I’ll be the first to admit that my own programming is as buggy as anybody else’s, but if I’m making life-and-death or world-changing decisions, I want to be sure that I’m not the only one who’s looked at my code, tested border cases, considered the implications of missing values, controlled for underflow and overflow errors, done smart things with floating point fuzziness, and generally thought about any given problem in a few more directions than I have. I want to know that when serious bugs are discovered, the knowledge will be disseminated and somebody’s job is on the line to fix them.

For all these reasons, I temper my sincere enthusiasm about the wide open frontiers of open source products like R with a conservative appreciation for software that has a big company’s reputation and future riding on its accuracy, and preferably a big company that has been in the business long enough to develop the paranoia that drives a fierce QA program.

R is great for what it is, as long as you bear in mind what it isn’t. Your own R code or R code that you find sitting around is only as good as your commitment to testing and understanding of thorny computational gotchas.

I share the apparently-common opinion that R’s interface leaves a lot to be desired. Confidentiality agreements prevent me from confirming or denying the rumors about JMP 9 interfacing with R, but I will say that if they turn out to be true, both products would benefit from it. JMP, like any commercial product, improves when it faces stiff competition and attends to it, and R, like most open source products, could use a better front end.

And now let me make my case for R being an epic success.

I like open source software. I use a bunch of it, and I do what I can for the cause (which isn’t much more than evangelism, unfortunately). For me, the biggest win with open source software is that it makes tools available to me, and others, who don’t need them enough to justify much of a price, but who can benefit from them when they’re affordable or free. When an open source tool gets something done for me, or eases some pain at least, I’m not that picky about its interface, and I’m willing to do my own validation (where applicable).

I can’t say that I love using Linux, but as a long-time UNIX geek and Mac OS X bigot, I am glad Linux is available, I use it for certain things, and I think it’s a whole lot better than Windows and other OSes, especially when Ubuntu builds work out. (I’ve had trouble getting JMP for Linux installed on Ubuntu, but that’s probably due to my own incompetence.) OpenOffice is kind of a pain, but it’s better than paying Microsoft for the privilege of enduring the epic fail that is Office, and it has much better support than Office for import/export of other formats. I love it that any number of open source projects are developing such fabulous tools as bzr version control, which I use daily, and that the FINK project is porting a whole bunch of great open source UNIX widgets to Mac OS X.

I think it’s wonderful that some of the world’s greatest analytical minds are using R to create publicly available routines for power-analysts. I love it that students and people who can’t afford commercial stats software, or who won’t use it enough to justify buying a license, have a high-quality open source option, if they’re willing to work at it a bit. I think it’s great that people who think Excel is good enough can’t make a price objection to upgrading to R.

I believe that democratizing innovation and proliferating analytical competence are good for us all. I count on projects like R and Linux to push commercial developers to make better products, and to force pricing and licensing of those products to remain reasonable. Monopolies are good for nobody, including monopolists.

Long live the proponents of R!

What do you think? Do you trust open source stats code? Do you think R’s interface is good enough? Is JMP’s any better? How heavily do you factor quality of documentation into decisions about software?

“Iterative Regression” is a script that Global Pragmatica LLC® developed for Ron Tanasichuk of the Canadian Department of Fisheries and Oceans, implementing his own interative regression technique. Performing the analysis is by hand is cumbersome and extremely slow and cumbersome. Automating the analysis reduces the processing time from weeks to minutes and eliminates the human error.

The method:

1. for a single Y response and multiple X factors, fit all possible models having one and two factors (no interactions)
2. for each model, exclude any outliers, and refit the model
3. repeat fitting and excluding until there are no outliers for the model
4. save the results from that model
5. re-include all rows (unexclude all rows) and begin again with the next factor combination

Outliers are defined as rows where Abs(Studentized residuals > 2.5) AND (Y-hats > 4/n).

The script is compatible with JMP 7, 8, and 9 and has been thoroughly tested on Mac versions JMP 7, 8, and 9. A version tested on Windows as well will be available shortly.

Buy now!

Contact me if you’re interested in purchasing this script or commissioning one like it.