Month: March 2024 (page 1 of 1)

Updatable Views

Welcome to a coffee break post where I quickly write up something on my mind that can be written and read in less time than a coffee break takes.

Recently, while exploring possible options for converting our existing ETLs to working with Postgres, I found myself asking the question “can you update a table using a view that has been created on that table?” and the answer is Yes for both Postgres and SQL Server, which I learned after doing some research.

In Postgres, there is a concept of an updatable view, which is essentially a very simple view, that usually doesn’t include a join to a second (or third or more) table. There are other qualifications for what makes a view updatable, but it is generally the fact that the query creating the view is very simple. If a view meets these criteria, you can in fact update the base table using a query to the view.

For example, if you have a table called employee, then you create a view on top of that table which selects most but not all the columns from the base table, with the view being called v_employee, you can then run this query and it will update the data in employee.

UPDATE v_employee
SET full_name = 'George Smith'
WHERE emp_id = 15;

This concept is intriguing to me, and could also be really useful in updating our ETLs more quickly after we convert to Postgres, so I thought I would share this fun little fact with everyone.

Sources

Postgres vs. SQL Server, Part 3

If you haven’t seen it already, I had two previous posts listing differences I have found between Postgres and SQL Server while learning to develop in the Postgres environment. As I am continuing to learn, I am finding even more differences that I think are interesting, and they are going to be shared below.

What’s in this post

Creating a temporary table

The difference between SQL Server and Postgres for creating a temp table is small, but significant. Mostly because my company uses temp tables frequently in our stored procedure and ETL code, so as we migrate off of SQL Server, we’re going to have to do a lot of updating to move temp table statements to Postgres. The syntax for SQL Server is to use a hashtag/pound/number symbol in front of the table name, or to use a double-pound symbol to create a global temporary table, like “CREATE TABLE #MyTable” or “CREATE TABLE ##MyGlobalTempTable“. In Postgres, the syntax requires that you use the TEMP or TEMPORARY keyword to make the temp table, like “CREATE TEMP[ORARY] TABLE my_table“.

You also have to use the TEMP or TEMPORARY keyword when creating a temp table in Postgres by doing a SELECT INTO statement, such as SELECT column1, column2 INTO TEMP TABLE my_table. In SQL Server, that same command would look like SELECT column1, column2 INTO #MyTable.

General differences between temp tables in each engine

Not only is the way you create temporary tables different between Postgres and SQL Server, but how the tables are actually stored on the database is completely different between the two as well.

In SQL Server, any temporary objects are stored in a separate database called tempdb. When creating a temporary table, you can specify whether it’s localized to your current session only or whether it is a global temporary table that can be accessed from different sessions or query windows, as I mentioned above. To do the first, you specify the table name as #MyTable, then the do the latter, you specify the table name as ##MyTable.

In Postgres, temporary tables are not stored in a separate database. Instead, they are put into their own system schemas whose names are based on numerical values (ex. pg_temp_1, pg_temp_2, etc.), so it’s harder to see the specific location where your temp table is being stored. You also are generally unable to access a temp table in one session from a different session, so there is no concept of a global temp table in Postgres.

One similarity of temp tables between the two engines, though, is that they are dropped/deleted in both as soon as the session in which they were created ends.

Getting the current UTC time

In SQL Server, there are two built-in functions that allow you to get the current UTC time, GETUTCDATE() and SYSUTCDATETIME(). The former returns the time in type DATETIME, and the latter returns the time in type DATETIME2 (more precise).

Postgres also has multiple functions and methods of getting the current UTC time, but they look much different from each other than the SQL Server options look. While there are more than just these two options, the most common ways to get the UTC time are the following:

  • SELECT NOW() AT TIME ZONE 'utc';
  • SELECT CURRENT_TIMESTAMP AT TIME ZONE 'utc'

Although the two options look different, they will return the same value. I’ve had some issues trying to use one versus the other in different scenarios in the SQL code I’ve written for Postgres, so it seems like there may be times where it’s better to use one over the other, but I’m just not able to differentiate those scenarios right now. There are also other options for getting the current UTC time, but I won’t cover those here.

Updating a table with a JOIN

One of the biggest syntax differences between T-SQL and pgSQL that has really gotten me confused so far is the formatting of an UPDATE statement that includes a JOIN to a different table. To me, the syntax of T-SQL makes a lot more logical sense and the pgSQL syntax doesn’t seem as intuitive. The T-SQL syntax is as follows:

UPDATE p
SET p.MyColumn = s.MyColumn
FROM person AS p
INNER JOIN staging AS s
	ON s.MyColumn2 = p.MyColumn2

Then the syntax for pgSQL is:

UPDATE person
SET p.my_column = s.my_column
FROM staging
WHERE staging.my_column2 = person.my_column2

To me, the pgSQL syntax doesn’t scream “joining to a different table”; it really looks to me like a simple update statement without a join. Yet the join is there. Also, it doesn’t seem to let you use aliases for the query, which I also dislike since I use aliases for every query I write. I wish I could change the standard for the pgSQL syntax, but obviously that isn’t within my power so I’m going to have to learn to live with it.

Making a column nullable

Unlike the pgSQL syntax for updating a table using a join to a different table, I actually like the syntax of making a column nullable in pgSQL more than I like the T-SQL syntax. The pgSQL syntax seems much more clear and straightforward for this action. The syntax in Postgres for making a NOT NULL column nullable is as follows: ALTER TABLE my_table ALTER COLUMN my_column DROP NOT NULL;

The T-SQL syntax for the same action is less clear, since you are basically redefining the column like you would if you were adding it, instead of specifying what you are changing. If you had a column that was set to NOT NULL in a table, you would write out the whole definition of the column again but write NULL instead of NOT NULL. For example: ALTER TABLE Mytable ALTER COLUMN MyColumn INT NULL;

I’m not fond of that T-SQL syntax because you can’t tell at a glance what is changing with the column definition unless you already knew what it was before running the statement. Since that same statement would also be the same syntax for changing the data type for the column if the column was already set to be nullable.

Data type for automatically updated binary numbers

In SQL Server, there is a data type called TIMESTAMP or ROWVERSION that is an automatically updating binary value that is added to a record when that record is updated or inserted. (The TIMESTAMP synonym for the ROWVERSION data type has been deprecated.) My company currently uses this data type heavily to help identify changed data quickly for our ETLs. As we are looking at moving everything into the Postgres world, we have found that Postgres does not have any built-in functionality similar to the ROWVERSION column, at least not at the precision we would need it to be (you can read about system columns here). You of course can store binary data within a table, but there is no data type for columns that will automatically increment a binary value when data is changed with the same precision as SQL Server’s ROWVERSION.

There do seem to be some ways people have replicated that functionality using custom triggers/functions that we are considering, but I haven’t looked into it that much personally so I cannot yet speak to how well that would work to replicate SQL Server’s ROWVERSION functionality. So we are either going to have to add these custom triggers/functions or find another reliable and fast way to identify changed data between source and target for our ETLs. Soon, I am hoping to do a blog post about the possible Change Data Capture (CDC) options for Postgres that we are considering.

Conclusion

There are so many weird quirks and differences between SQL Server and Postgres that I am having to learn as I grow in the Postgres development space. While I have already listed so many of them across several posts now, I am sure there will be many more to come as I keep learning. And I will continue to compile a list of the differences and share them here on my blog as I find them.

Sources

  • https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-temporary-table/
  • https://www.postgresql.org/docs/current/datatype-datetime.html
  • https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-update-join/
  • https://www.postgresql.org/docs/7.3/sql-altertable.html

Getting Better at Keeping Work at Work

Despite already writing a similar post about this topic myself, I have still been struggling recently to keep my work at work, struggling with not thinking about it in my free time in the evenings and on weekends. Looking for some professional guidance from an outside perspective online, I found this great article from Harvard Business Review that I want to share with others.

Using Technology to Log Out for the Day

Steps 1 and 3 from the article come easily to me. While I don’t have specific hours I am required to be at work, I am great at keeping a regular work schedule for myself that I don’t normally deviate from. Once I log out at 5 PM each day, I don’t check Teams messages, and I don’t even have my work email on my phone so I can’t check that even if I wanted to. I manage this system using the Focus Time feature on my iPhone.

I have two different scheduled Focus Time settings for my phone: 1) Work and 2) Personal Time. I’ve set up the Work Focus Time to be active during my normal work hours and it only allows notifications from Teams, my two-factor authentication apps, and a couple of important family members. Then I’ve set up the Personal Time custom Focus Time to be the opposite: it runs from the time I get off work to the time I start work in the morning and blocks any notifications from Teams or other work-related applications.

This system has been working really well for me and I would recommend others try it out if they haven’t already, because then you never need to think about which notifications to turn off after work each day, or worse, just keep getting those notifications all evening and weekend long that take your mind back to work when it shouldn’t be there.

Struggling with “Mental Clarity” Around Work

Steps 2 and 4 from the HBR article are more challenging for me lately than I would like them to be. Step 2 is to get “mental clarity”, which essentially means that you should know each day what tasks you need to accomplish at work, by keeping a running to-do list or somewhere else to manage all your thoughts about work. I used to be great at this before the last couple months, when my job became less clear and began changing. I stopped keeping a physical notebook because I needed a new system to keep related notes related and physically close since I now have larger projects to manage rather than small tickets to work on daily or weekly. Once I stopped heavily utilizing my physical notebook, I stopped creating daily to-do lists for myself to accomplish.

Not having a go-to place to list out all the current and future things I will need to complete has led to the problem of me randomly remembering important things I need to do for work at inconvenient times, like when I’m trying to fall asleep. I’m sure many can relate to that. This mental clarity step reminds me of the book “Getting Things Done” by David Allen that I read a couple years ago. One of the main points of that book is you need to be constantly dumping thoughts out of your head onto paper (or anywhere you want as long as it’s consistent) so your brain can trust it doesn’t need to constantly remind you of important things at inappropriate times. If you know that you always write important tasks in the same location, you know that anything important is in that location and not just in your head. The Harvard Business Review article made me realize I need to get back to that organization method and start actually using the digital note page I made awhile ago that is supposed to act as my catch-all list of items on my plate now and in the future.

One final note on the mental clarity step of keeping work at work– I actually do still manage to complete an end-of-day wrap up each work day where I write out everything I accomplished throughout the day along with notes for what I need to do the next day and a review of the next day’s calendar. I occasionally skip days of this, so I would like to schedule time for this each day going forward so I’m more diligent about doing it.

The Need to Feel Accomplished

The final step to keeping work at work recommended in the article that I’ve been struggling with the most recently is step 4, which says that you need to get work done at work. It may seem crazy for someone to not get work done during their work day if they’re showing up on time and leaving usually later than they should, but it is entirely possible if you allow your entire schedule to be consumed by meetings with no or minimal time between them. My job has morphed into a collaboration-focused position where I suddenly need to have meetings upon meetings about everything related to the projects I am working on. This means that on an average day, I have multiple hours of meetings with colleagues, and somehow they all seem to leave 30-minute gaps between to tease me with the possibility of getting work done, but then I never actually get time to focus in those gaps.

This cycle of continual meetings and no long stretches of work time has made it really difficult for me to have any decent amount of time to focus on the deliverables I need to complete. While sure, you could theoretically get good work done in the 30 minutes between meetings, reality is much different when you need to work on a confusing or difficult task. Those types of work items are better suited for longer stretches of dedicated focus time, at least one hour but hopefully two or more. That is a rare occurrence in my current work life and I want to change that, and will do so by following the recommendations of this HBR article, and actually some of my own advice from a previous blog post. I need to schedule blocks of one or more hours each day to focus on work. My plan is to do that on Friday afternoons the week before or Monday mornings the week of, so I can block out work time for an entire week to get ahead of meetings that will inevitably be scheduled.

Summary

While the advice of the Harvard Business Review article isn’t entirely new to me, it did act as a great trigger to remind me of the valuable work skills I have neglected recently so I can revive them. Going forward, I am going to revitalize my use of my own calendar and digit notebook to see if that helps me feel more in control and less caught up in the whirlwind at work.

Give the HBR article a read and see if any of it speaks to you like it spoke to me. Is there something in your work-life balance situation that could be improved to reduce your overall stress?

Getting SQL Prompt to Prompt on RDS Servers

This may seem like a ridiculous thing to need to write about, making the Red-Gate tool SQL Prompt generate prompts like it should, but I have been having a weird issue with it over the past couple months and have finally learned the solution. So of course I thought I should share it!

What is SQL Prompt?

SQL Prompt is a tool made by Red-Gate that works as a much cleaner, nicer, and more useful autocomplete feature for SQL Server Management Studio (SSMS). It is a plugin you install to SSMS that then seems to magically work to help you write queries faster. Not only does this tool autocomplete databases, schemas, tables, and column names for you in your queries, but it also provides a lot of other useful tools like a Snippets Manager, which allows you to use default and custom snippets to write code faster (e.g. writing “sf” then pressing Tab will type out “SELECT * FROM” for you so all you need to type is the table name you want to select from).

Every developer in my organization uses this tool heavily in our day-to-day operations while writing any SQL scripts because it makes writing queries so much faster. So when my SQL Prompt seemed to stop working after an update, I was getting really frustrated because it meant I had to write all of my SQL queries manually again. And when all you do all day is write SQL, that adds up to a significant hindrance to your work speed.

My Problem

The problem I was having with SQL Prompt was that when I connected to any of our RDS database instances, the tool would no longer do any prompting of schemas, tables, or columns which was making my coding life so much harder. Oddly, the snippets manager portion of the tool was still working fine, so at least I wasn’t needing to type out the queries I normally use the snippets shortcuts for. Also oddly, Prompt would work perfectly fine connecting to other databases that weren’t on RDS instances, it was only happening for RDS databases.

I dealt with this issue for months on our production server, since I figured it was due to the security settings or something else I wouldn’t be able to fix, and I don’t access prod servers very frequently, so when I did use them and the Prompt wasn’t working, it wasn’t as bothersome. But after I had to completely reset my developer computer and reinstalled SQL Prompt, I started having this same issue for our lower environment databases, that I work with every day, so Prompt not prompting was suddenly a big deal.

The Solution

I created a support ticket with Red-Gate since I’ve always had good luck with their support services. This time wasn’t any different. Within a couple hours of creating the ticket, I had an email from a support rep asking me if I had tried checking the “Trust Server Certificate” check box on the Connection Properties tab while connecting to the RDS servers. No, I had not done that because I did not know that was an option before that day.

I disconnected from the RDS server then reconnected, making sure to check that box before clicking “Connect”, and now I had SQL Prompt back up and working, providing prompts of schemas, tables, and columns just like I want it to. Yay!

I also logged onto our prod server to see if doing the same thing there would fix that issue, and it fixed Prompt there as well. I am so excited to be able to not type out every detail of every SQL query again!

I love quick fixes like this.