hooking into sql server file I/O -- file level encryption

There are a number of third party packages that do "whole file"
encryption of SQL databases. For example:
Encryptionizer (http://www.netlib.com/)
XP_CRYPT (http://www.xpcrypt.com/)
We are considering using one of these products, but I am wondering if
it is possible to "roll our own". We have some experience doing this
with sqlite; because sqlite is open source, it is relatively easy to
hook into the "pager" and perform custom encryption/decryption.
What I'd like to know is whether something similar is possible with
SQL Server (or more specifically MSDE). Or rather, clearly it is
_possible_, since the above products do it. But how? How do these
products insert themselves between the database engine and the
physical file I/O?
I speculate that they are re-implementing some file IO interfaces used
by SQL, but I don't know where to start looking for what those
interfaces might be or where they live in the registry. Or maybe I am
off base and there are some hooks provided by SQL I don't know about.
I have scoured MSDN and Google but it appears to be a difficult topic
to query.
Does anyone have any idea how this works?
TIA,
Rol(rolandp66@.yahoo.com) writes:
> There are a number of third party packages that do "whole file"
> encryption of SQL databases. For example:
> Encryptionizer (http://www.netlib.com/)
> XP_CRYPT (http://www.xpcrypt.com/)
> We are considering using one of these products, but I am wondering if
> it is possible to "roll our own". We have some experience doing this
> with sqlite; because sqlite is open source, it is relatively easy to
> hook into the "pager" and perform custom encryption/decryption.
> What I'd like to know is whether something similar is possible with
> SQL Server (or more specifically MSDE). Or rather, clearly it is
> _possible_, since the above products do it. But how? How do these
> products insert themselves between the database engine and the
> physical file I/O?
> I speculate that they are re-implementing some file IO interfaces used
> by SQL, but I don't know where to start looking for what those
> interfaces might be or where they live in the registry. Or maybe I am
> off base and there are some hooks provided by SQL I don't know about.
> I have scoured MSDN and Google but it appears to be a difficult topic
> to query.
The way to do it would indeed be to intercept the file I/O system calls.
Since SQL Server uses the interfaces in NTFS, the tool would have to
be configured to only encrypt selected files, since else you would be
encrypting - and deenscrypting - the entire disk.
Whether this actually is doable I don't know. You would have to ask in
a Windows newsgroup to get some tips on that. But I would assume that
whatever you do, it would be entirely unsupported.
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx|||I may be missing something basic here but wouldn't it be a lot easier to
just use NTFS encryption to encrypt the files? One checkbox in the file
attributes seems a lot easier than writing your own disk driver.
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm
<rolandp66@.yahoo.com> wrote in message
news:1175719135.450235.272090@.b75g2000hsg.googlegroups.com...
> There are a number of third party packages that do "whole file"
> encryption of SQL databases. For example:
> Encryptionizer (http://www.netlib.com/)
> XP_CRYPT (http://www.xpcrypt.com/)
> We are considering using one of these products, but I am wondering if
> it is possible to "roll our own". We have some experience doing this
> with sqlite; because sqlite is open source, it is relatively easy to
> hook into the "pager" and perform custom encryption/decryption.
> What I'd like to know is whether something similar is possible with
> SQL Server (or more specifically MSDE). Or rather, clearly it is
> _possible_, since the above products do it. But how? How do these
> products insert themselves between the database engine and the
> physical file I/O?
> I speculate that they are re-implementing some file IO interfaces used
> by SQL, but I don't know where to start looking for what those
> interfaces might be or where they live in the registry. Or maybe I am
> off base and there are some hooks provided by SQL I don't know about.
> I have scoured MSDN and Google but it appears to be a difficult topic
> to query.
> Does anyone have any idea how this works?
> TIA,
> Rol
>|||Roger Wolter[MSFT] (rwolter@.online.microsoft.com) writes:
> I may be missing something basic here but wouldn't it be a lot easier to
> just use NTFS encryption to encrypt the files? One checkbox in the file
> attributes seems a lot easier than writing your own disk driver.
Just for fun, I stopped my SQL 2000 instance and then encrypted the
Northwind database from Explorer. When I started SQL Server, I found that
I was not able to access Northwind. This was what the SQL Server had to
tell me:
2007-04-07 11:10:32.17 spid9 Device activation error. The physical file
name 'E:\Program Files\Microsoft SQL Server\MSSQL\data\northwnd.mdf' may be
incorrect.
2007-04-07 11:10:32.31 spid9 udopen: Operating system error 5(error not
found) during the creation/opening of physical device E:\Program Files\Micro
soft SQL Server\MSSQL\data\northwnd.ldf.
2007-04-07 11:10:32.31 spid9 FCB::Open failed: Could not open device E:\
Program Files\Microsoft SQL Server\MSSQL\data\northwnd.ldf for virtual devic
e number (VDN) 2.
Now, I know zero about NTFS encryption, so there may be a simple solution,
but I figured that I should make this note, in case someone would get the
idea to try this for real.
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspx|||The key is associated with the user who does the encrypting so if you're not
encrypting while logged in as the domain user that sqlservr.exe executes as,
SQL won't be able to read the file.
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm
"Erland Sommarskog" <esquel@.sommarskog.se> wrote in message
news:Xns990B72C517060Yazorman@.127.0.0.1...
> Roger Wolter[MSFT] (rwolter@.online.microsoft.com) writes:
> Just for fun, I stopped my SQL 2000 instance and then encrypted the
> Northwind database from Explorer. When I started SQL Server, I found that
> I was not able to access Northwind. This was what the SQL Server had to
> tell me:
> 2007-04-07 11:10:32.17 spid9 Device activation error. The physical
> file name 'E:\Program Files\Microsoft SQL Server\MSSQL\data\northwnd.mdf'
> may be incorrect.
> 2007-04-07 11:10:32.31 spid9 udopen: Operating system error 5(error
> not found) during the creation/opening of physical device E:\Program
> Files\Microsoft SQL Server\MSSQL\data\northwnd.ldf.
> 2007-04-07 11:10:32.31 spid9 FCB::Open failed: Could not open device
> E:\Program Files\Microsoft SQL Server\MSSQL\data\northwnd.ldf for virtual
> device number (VDN) 2.
>
> Now, I know zero about NTFS encryption, so there may be a simple solution,
> but I figured that I should make this note, in case someone would get the
> idea to try this for real.
>
> --
> Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
> Books Online for SQL Server 2005 at
> http://www.microsoft.com/technet/pr...oads/books.mspx
> Books Online for SQL Server 2000 at
> http://www.microsoft.com/sql/prodin...ions/books.mspx|||Roger Wolter[MSFT] (rwolter@.online.microsoft.com) writes:
> The key is associated with the user who does the encrypting so if you're
> not encrypting while logged in as the domain user that sqlservr.exe
> executes as, SQL won't be able to read the file.
I suspected it was something like that, but I didn't feel like testing it.
Thanks for straighten it out, Roger.
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pr...oads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodin...ions/books.mspxsql

High performance isn't always an advantage...

Hello
MSSQL 2000 + SP3 on Windows 2000 AS + SP4
I've moved my databases to new server.
IO subsystem now TEN times faster!
I have the system of two jobs performing defragmentation
of my data. One of the jobs issues DBCC INDEXDEFRAG
statements and the second one monitors state of transaction
log and stops first job if log more than 70% full. SQLAgent
can run jobs only with 1 minute intervals. So I can monitor
log status only once in minute. Everything was all right at old
server. Now on new hardware log overfills in couple of seconds
and monitoring job can't catch the moment of overfilling and
correctly stop the defragmentation.
Two questions:
1. Should I increase Tlog size depending on hardware performance
2. Is there any way to monitor log state more ofter than once
in a minute?
Thanks.
Serge ShakhovYou can have a job which uses an endless loop and WAITFOR with, say, 10 seconds.
--
Tibor Karaszi, SQL Server MVP
Archive at: http://groups.google.com/groups?oi=djq&as ugroup=microsoft.public.sqlserver
"Serge Shakhov" <REMOVETHIS_ACETYLENE@.mail.ru> wrote in message
news:83ndmb.5kr.ln@.proxyserver.ctd.mmk.chel.su...
> Hello
> MSSQL 2000 + SP3 on Windows 2000 AS + SP4
> I've moved my databases to new server.
> IO subsystem now TEN times faster!
> I have the system of two jobs performing defragmentation
> of my data. One of the jobs issues DBCC INDEXDEFRAG
> statements and the second one monitors state of transaction
> log and stops first job if log more than 70% full. SQLAgent
> can run jobs only with 1 minute intervals. So I can monitor
> log status only once in minute. Everything was all right at old
> server. Now on new hardware log overfills in couple of seconds
> and monitoring job can't catch the moment of overfilling and
> correctly stop the defragmentation.
> Two questions:
> 1. Should I increase Tlog size depending on hardware performance
> 2. Is there any way to monitor log state more ofter than once
> in a minute?
> Thanks.
> Serge Shakhov
>|||Hello
> You can have a job which uses an endless loop and WAITFOR with, say, 10
seconds.
That's a good idea!
But I'll try to avoid cycling process by executing every minute job wich
will last 58-59 seconds and perform checks say every 5 seconds...
Thank you Tibor.
Serge Shakhovsql

High number of Page Faults

Svein Terje Gaup wrote:
> Hi, I have a SQL Server 2000 with 14 different databases. I have been
> monitoring it for some time, and it seems there is a high number of
> page faults occurring. The number is constantly above 3000. The server
> has 3 Gigabytes of RAM, and there's about 600+ megs free at all times.
> The Pages/sec counter is stabilised at a very low level (mostly it is
> at 0). The SQL Server Cache hit ratio is mostly at 99%. The server is a
> dedicated SQL Server.
> So what could be causing the high amount of Page Faults?
> Sincerely
> Svein Terje Gaup
>
Do you have something copying files off of this machine, perhaps a tape
backup or something copy files across the network? I see this sort of
thing when copying files from server to server. The page faults occur
as the files are read from disk.Hi, I have a SQL Server 2000 with 14 different databases. I have been
monitoring it for some time, and it seems there is a high number of
page faults occurring. The number is constantly above 3000. The server
has 3 Gigabytes of RAM, and there's about 600+ megs free at all times.
The Pages/sec counter is stabilised at a very low level (mostly it is
at 0). The SQL Server Cache hit ratio is mostly at 99%. The server is a
dedicated SQL Server.
So what could be causing the high amount of Page Faults?
Sincerely
Svein Terje Gaup|||Svein Terje Gaup wrote:
> Hi, I have a SQL Server 2000 with 14 different databases. I have been
> monitoring it for some time, and it seems there is a high number of
> page faults occurring. The number is constantly above 3000. The server
> has 3 Gigabytes of RAM, and there's about 600+ megs free at all times.
> The Pages/sec counter is stabilised at a very low level (mostly it is
> at 0). The SQL Server Cache hit ratio is mostly at 99%. The server is a
> dedicated SQL Server.
> So what could be causing the high amount of Page Faults?
> Sincerely
> Svein Terje Gaup
>
Do you have something copying files off of this machine, perhaps a tape
backup or something copy files across the network? I see this sort of
thing when copying files from server to server. The page faults occur
as the files are read from disk.|||Thank you for your answer. I will try to look into it, and see if any
jobs are running that copies files from the server.
The Page Faults are constantly over 3000 during the whole working day.
Maybe there's some fileshare that is accessed constantly by users (it's
a customer's server, and they said it was dedicated as a sql server,
but who knows...) ? Anyway, I will get back with any results.
Sincerely
Svein Terje Gaup
Tracy McKibben wrote:
> Svein Terje Gaup wrote:
> Do you have something copying files off of this machine, perhaps a tape
> backup or something copy files across the network? I see this sort of
> thing when copying files from server to server. The page faults occur
> as the files are read from disk.|||Thank you for your answer. I will try to look into it, and see if any
jobs are running that copies files from the server.
The Page Faults are constantly over 3000 during the whole working day.
Maybe there's some fileshare that is accessed constantly by users (it's
a customer's server, and they said it was dedicated as a sql server,
but who knows...) ? Anyway, I will get back with any results.
Sincerely
Svein Terje Gaup
Tracy McKibben wrote:
> Svein Terje Gaup wrote:
> Do you have something copying files off of this machine, perhaps a tape
> backup or something copy files across the network? I see this sort of
> thing when copying files from server to server. The page faults occur
> as the files are read from disk.|||I'd try to find out what process(es) is primarily responsible for these page
faults. Go to the Processes tab on Task Manager, and rank the output by page
Faults or PF delta. It's common to have page faults in hundreds without any
performance issue. But if it's constantly around 3000, I'd conisder that hig
h
(just from the numbers I've seen on some of the busy servers in my
environments).
Linchi
"Svein Terje Gaup" wrote:

> Thank you for your answer. I will try to look into it, and see if any
> jobs are running that copies files from the server.
> The Page Faults are constantly over 3000 during the whole working day.
> Maybe there's some fileshare that is accessed constantly by users (it's
> a customer's server, and they said it was dedicated as a sql server,
> but who knows...) ? Anyway, I will get back with any results.
> Sincerely
> Svein Terje Gaup
>
> Tracy McKibben wrote:
>|||I'd try to find out what process(es) is primarily responsible for these page
faults. Go to the Processes tab on Task Manager, and rank the output by page
Faults or PF delta. It's common to have page faults in hundreds without any
performance issue. But if it's constantly around 3000, I'd conisder that hig
h
(just from the numbers I've seen on some of the busy servers in my
environments).
Linchi
"Svein Terje Gaup" wrote:

> Thank you for your answer. I will try to look into it, and see if any
> jobs are running that copies files from the server.
> The Page Faults are constantly over 3000 during the whole working day.
> Maybe there's some fileshare that is accessed constantly by users (it's
> a customer's server, and they said it was dedicated as a sql server,
> but who knows...) ? Anyway, I will get back with any results.
> Sincerely
> Svein Terje Gaup
>
> Tracy McKibben wrote:
>|||Thank you. Thats a good idea. I'll try it next time I'm on-site.
Sincerely
Svein Terje Gaup
Linchi Shea wrote:[vbcol=seagreen]
> I'd try to find out what process(es) is primarily responsible for these pa
ge
> faults. Go to the Processes tab on Task Manager, and rank the output by pa
ge
> Faults or PF delta. It's common to have page faults in hundreds without an
y
> performance issue. But if it's constantly around 3000, I'd conisder that h
igh
> (just from the numbers I've seen on some of the busy servers in my
> environments).
> Linchi
> "Svein Terje Gaup" wrote:
>

High number of Page Faults

Hi, I have a SQL Server 2000 with 14 different databases. I have been
monitoring it for some time, and it seems there is a high number of
page faults occurring. The number is constantly above 3000. The server
has 3 Gigabytes of RAM, and there's about 600+ megs free at all times.
The Pages/sec counter is stabilised at a very low level (mostly it is
at 0). The SQL Server Cache hit ratio is mostly at 99%. The server is a
dedicated SQL Server.
So what could be causing the high amount of Page Faults?
Sincerely
Svein Terje GaupSvein Terje Gaup wrote:
> Hi, I have a SQL Server 2000 with 14 different databases. I have been
> monitoring it for some time, and it seems there is a high number of
> page faults occurring. The number is constantly above 3000. The server
> has 3 Gigabytes of RAM, and there's about 600+ megs free at all times.
> The Pages/sec counter is stabilised at a very low level (mostly it is
> at 0). The SQL Server Cache hit ratio is mostly at 99%. The server is a
> dedicated SQL Server.
> So what could be causing the high amount of Page Faults?
> Sincerely
> Svein Terje Gaup
>
Do you have something copying files off of this machine, perhaps a tape
backup or something copy files across the network? I see this sort of
thing when copying files from server to server. The page faults occur
as the files are read from disk.|||Thank you for your answer. I will try to look into it, and see if any
jobs are running that copies files from the server.
The Page Faults are constantly over 3000 during the whole working day.
Maybe there's some fileshare that is accessed constantly by users (it's
a customer's server, and they said it was dedicated as a sql server,
but who knows...) ? Anyway, I will get back with any results.
Sincerely
Svein Terje Gaup
Tracy McKibben wrote:
> Svein Terje Gaup wrote:
> > Hi, I have a SQL Server 2000 with 14 different databases. I have been
> > monitoring it for some time, and it seems there is a high number of
> > page faults occurring. The number is constantly above 3000. The server
> > has 3 Gigabytes of RAM, and there's about 600+ megs free at all times.
> > The Pages/sec counter is stabilised at a very low level (mostly it is
> > at 0). The SQL Server Cache hit ratio is mostly at 99%. The server is a
> > dedicated SQL Server.
> >
> > So what could be causing the high amount of Page Faults?
> >
> > Sincerely
> > Svein Terje Gaup
> >
> Do you have something copying files off of this machine, perhaps a tape
> backup or something copy files across the network? I see this sort of
> thing when copying files from server to server. The page faults occur
> as the files are read from disk.

High Memory Usage by SQL 2000

Hi,
Recently i've moved some of our databases from a Windows 2000 Server with
512 physical memory to a newer Windows 2003 Server with 1024MB physical
memory.
The new server functions OK, but there is a very high usage of memory in
comparison to the old server: the sqlservr.exe process used around 400MB of
memory,
while the new one is using near 850MB after only 2 weeks (while at the
beginning it used something like 600 MB).
Is such a behaviour is normal and can be explained by that the new server
has larger available amount of physical memory?
In any case, are there any tools that helps to determine which application
catch the largest amount of memory used by SQL?
I think that such a large usage for relatively no so hard-working server is
something strange.
Any help/tips will be highly appreciated!!!
Thanks in advance,
BarakBarak,
SQL Server 2000 will grab and hold on to as much memory as it needs. SQL
Server 2000, will not, by default, release memory back to the operating
system after using it. This is normal and expected behavior. The basic
reason is that Microsoft expects that SQL Server 2000 is installed as a
stand-alone application on a server. Thus not only is nothing gained by SQL
Server 2000 releasing memory back to the OS, but, in fact, performance is
hurt if SQL Server 2000 was continually requesting and releasing memory ...
the request for and release of memory is fairly obviously an activity that
requires some resources, at the very least CPU time, that could otherwise be
utilized.
Additional information is available in the SQL Server 2000 Books Online in
sections "Memory Architecture" and "Dynamically Managing Memory on Windows
NT and Windows 2000".
Chief Tenaya
"Barak Turovsky" <baraktur@.mail.ru> wrote in message
news:uslssSIHEHA.1432@.TK2MSFTNGP12.phx.gbl...
> Hi,
> Recently i've moved some of our databases from a Windows 2000 Server with
> 512 physical memory to a newer Windows 2003 Server with 1024MB physical
> memory.
> The new server functions OK, but there is a very high usage of memory in
> comparison to the old server: the sqlservr.exe process used around 400MB
of
> memory,
> while the new one is using near 850MB after only 2 weeks (while at the
> beginning it used something like 600 MB).
> Is such a behaviour is normal and can be explained by that the new server
> has larger available amount of physical memory?
> In any case, are there any tools that helps to determine which application
> catch the largest amount of memory used by SQL?
> I think that such a large usage for relatively no so hard-working server
is
> something strange.
> Any help/tips will be highly appreciated!!!
> Thanks in advance,
> Barak
>|||Barak,
BTW, within limits, the more memory available to SQL Server 2000, the better
one can expect performance from SQL Server 2000. Additional memory allows
SQL Server 2000 to create and use a larger memory cache area, thus reducing
physical I/O ... and physical I/O is more often then not what drags down
performance in most any database.
Chief Tenaya
"Barak Turovsky" <baraktur@.mail.ru> wrote in message
news:uslssSIHEHA.1432@.TK2MSFTNGP12.phx.gbl...
> Hi,
> Recently i've moved some of our databases from a Windows 2000 Server with
> 512 physical memory to a newer Windows 2003 Server with 1024MB physical
> memory.
> The new server functions OK, but there is a very high usage of memory in
> comparison to the old server: the sqlservr.exe process used around 400MB
of
> memory,
> while the new one is using near 850MB after only 2 weeks (while at the
> beginning it used something like 600 MB).
> Is such a behaviour is normal and can be explained by that the new server
> has larger available amount of physical memory?
> In any case, are there any tools that helps to determine which application
> catch the largest amount of memory used by SQL?
> I think that such a large usage for relatively no so hard-working server
is
> something strange.
> Any help/tips will be highly appreciated!!!
> Thanks in advance,
> Barak
>

High Memory Usage by SQL 2000

Hi,
Recently i've moved some of our databases from a Windows 2000 Server with
512 physical memory to a newer Windows 2003 Server with 1024MB physical
memory.
The new server functions OK, but there is a very high usage of memory in
comparison to the old server: the sqlservr.exe process used around 400MB of
memory,
while the new one is using near 850MB after only 2 weeks (while at the
beginning it used something like 600 MB).
Is such a behaviour is normal and can be explained by that the new server
has larger available amount of physical memory?
In any case, are there any tools that helps to determine which application
catch the largest amount of memory used by SQL?
I think that such a large usage for relatively no so hard-working server is
something strange.
Any help/tips will be highly appreciated!!!
Thanks in advance,
BarakBarak,
SQL Server 2000 will grab and hold on to as much memory as it needs. SQL
Server 2000, will not, by default, release memory back to the operating
system after using it. This is normal and expected behavior. The basic
reason is that Microsoft expects that SQL Server 2000 is installed as a
stand-alone application on a server. Thus not only is nothing gained by SQL
Server 2000 releasing memory back to the OS, but, in fact, performance is
hurt if SQL Server 2000 was continually requesting and releasing memory ...
the request for and release of memory is fairly obviously an activity that
requires some resources, at the very least CPU time, that could otherwise be
utilized.
Additional information is available in the SQL Server 2000 Books Online in
sections "Memory Architecture" and "Dynamically Managing Memory on Windows
NT and Windows 2000".
Chief Tenaya
"Barak Turovsky" <baraktur@.mail.ru> wrote in message
news:uslssSIHEHA.1432@.TK2MSFTNGP12.phx.gbl...
> Hi,
> Recently i've moved some of our databases from a Windows 2000 Server with
> 512 physical memory to a newer Windows 2003 Server with 1024MB physical
> memory.
> The new server functions OK, but there is a very high usage of memory in
> comparison to the old server: the sqlservr.exe process used around 400MB
of
> memory,
> while the new one is using near 850MB after only 2 weeks (while at the
> beginning it used something like 600 MB).
> Is such a behaviour is normal and can be explained by that the new server
> has larger available amount of physical memory?
> In any case, are there any tools that helps to determine which application
> catch the largest amount of memory used by SQL?
> I think that such a large usage for relatively no so hard-working server
is
> something strange.
> Any help/tips will be highly appreciated!!!
> Thanks in advance,
> Barak
>|||Barak,
BTW, within limits, the more memory available to SQL Server 2000, the better
one can expect performance from SQL Server 2000. Additional memory allows
SQL Server 2000 to create and use a larger memory cache area, thus reducing
physical I/O ... and physical I/O is more often then not what drags down
performance in most any database.
Chief Tenaya
"Barak Turovsky" <baraktur@.mail.ru> wrote in message
news:uslssSIHEHA.1432@.TK2MSFTNGP12.phx.gbl...
> Hi,
> Recently i've moved some of our databases from a Windows 2000 Server with
> 512 physical memory to a newer Windows 2003 Server with 1024MB physical
> memory.
> The new server functions OK, but there is a very high usage of memory in
> comparison to the old server: the sqlservr.exe process used around 400MB
of
> memory,
> while the new one is using near 850MB after only 2 weeks (while at the
> beginning it used something like 600 MB).
> Is such a behaviour is normal and can be explained by that the new server
> has larger available amount of physical memory?
> In any case, are there any tools that helps to determine which application
> catch the largest amount of memory used by SQL?
> I think that such a large usage for relatively no so hard-working server
is
> something strange.
> Any help/tips will be highly appreciated!!!
> Thanks in advance,
> Barak
>

High Memory Usage by SQL 2000

Hi,
Recently i've moved some of our databases from a Windows 2000 Server with
512 physical memory to a newer Windows 2003 Server with 1024MB physical
memory.
The new server functions OK, but there is a very high usage of memory in
comparison to the old server: the sqlservr.exe process used around 400MB of
memory,
while the new one is using near 850MB after only 2 weeks (while at the
beginning it used something like 600 MB).
Is such a behaviour is normal and can be explained by that the new server
has larger available amount of physical memory?
In any case, are there any tools that helps to determine which application
catch the largest amount of memory used by SQL?
I think that such a large usage for relatively no so hard-working server is
something strange.
Any help/tips will be highly appreciated!!!
Thanks in advance,
Barak
Barak,
SQL Server 2000 will grab and hold on to as much memory as it needs. SQL
Server 2000, will not, by default, release memory back to the operating
system after using it. This is normal and expected behavior. The basic
reason is that Microsoft expects that SQL Server 2000 is installed as a
stand-alone application on a server. Thus not only is nothing gained by SQL
Server 2000 releasing memory back to the OS, but, in fact, performance is
hurt if SQL Server 2000 was continually requesting and releasing memory ...
the request for and release of memory is fairly obviously an activity that
requires some resources, at the very least CPU time, that could otherwise be
utilized.
Additional information is available in the SQL Server 2000 Books Online in
sections "Memory Architecture" and "Dynamically Managing Memory on Windows
NT and Windows 2000".
Chief Tenaya
"Barak Turovsky" <baraktur@.mail.ru> wrote in message
news:uslssSIHEHA.1432@.TK2MSFTNGP12.phx.gbl...
> Hi,
> Recently i've moved some of our databases from a Windows 2000 Server with
> 512 physical memory to a newer Windows 2003 Server with 1024MB physical
> memory.
> The new server functions OK, but there is a very high usage of memory in
> comparison to the old server: the sqlservr.exe process used around 400MB
of
> memory,
> while the new one is using near 850MB after only 2 weeks (while at the
> beginning it used something like 600 MB).
> Is such a behaviour is normal and can be explained by that the new server
> has larger available amount of physical memory?
> In any case, are there any tools that helps to determine which application
> catch the largest amount of memory used by SQL?
> I think that such a large usage for relatively no so hard-working server
is
> something strange.
> Any help/tips will be highly appreciated!!!
> Thanks in advance,
> Barak
>
|||Barak,
BTW, within limits, the more memory available to SQL Server 2000, the better
one can expect performance from SQL Server 2000. Additional memory allows
SQL Server 2000 to create and use a larger memory cache area, thus reducing
physical I/O ... and physical I/O is more often then not what drags down
performance in most any database.
Chief Tenaya
"Barak Turovsky" <baraktur@.mail.ru> wrote in message
news:uslssSIHEHA.1432@.TK2MSFTNGP12.phx.gbl...
> Hi,
> Recently i've moved some of our databases from a Windows 2000 Server with
> 512 physical memory to a newer Windows 2003 Server with 1024MB physical
> memory.
> The new server functions OK, but there is a very high usage of memory in
> comparison to the old server: the sqlservr.exe process used around 400MB
of
> memory,
> while the new one is using near 850MB after only 2 weeks (while at the
> beginning it used something like 600 MB).
> Is such a behaviour is normal and can be explained by that the new server
> has larger available amount of physical memory?
> In any case, are there any tools that helps to determine which application
> catch the largest amount of memory used by SQL?
> I think that such a large usage for relatively no so hard-working server
is
> something strange.
> Any help/tips will be highly appreciated!!!
> Thanks in advance,
> Barak
>
|||Chief Tenaya is correct and gave you some good references. You should also
take a look at 321363 INF: SQL Server Memory Usage
http://support.microsoft.com/?id=321363.
Cindy Gross, MCDBA, MCSE
http://cindygross.tripod.com
This posting is provided "AS IS" with no warranties, and confers no rights.

High Memory Usage by SQL 2000

Hi,
Recently i've moved some of our databases from a Windows 2000 Server with
512 physical memory to a newer Windows 2003 Server with 1024MB physical
memory.
The new server functions OK, but there is a very high usage of memory in
comparison to the old server: the sqlservr.exe process used around 400MB of
memory,
while the new one is using near 850MB after only 2 weeks (while at the
beginning it used something like 600 MB).
Is such a behaviour is normal and can be explained by that the new server
has larger available amount of physical memory?
In any case, are there any tools that helps to determine which application
catch the largest amount of memory used by SQL?
I think that such a large usage for relatively no so hard-working server is
something strange.
Any help/tips will be highly appreciated!!!
Thanks in advance,
Barak
Barak,
SQL Server 2000 will grab and hold on to as much memory as it needs. SQL
Server 2000, will not, by default, release memory back to the operating
system after using it. This is normal and expected behavior. The basic
reason is that Microsoft expects that SQL Server 2000 is installed as a
stand-alone application on a server. Thus not only is nothing gained by SQL
Server 2000 releasing memory back to the OS, but, in fact, performance is
hurt if SQL Server 2000 was continually requesting and releasing memory ...
the request for and release of memory is fairly obviously an activity that
requires some resources, at the very least CPU time, that could otherwise be
utilized.
Additional information is available in the SQL Server 2000 Books Online in
sections "Memory Architecture" and "Dynamically Managing Memory on Windows
NT and Windows 2000".
Chief Tenaya
"Barak Turovsky" <baraktur@.mail.ru> wrote in message
news:uslssSIHEHA.1432@.TK2MSFTNGP12.phx.gbl...
> Hi,
> Recently i've moved some of our databases from a Windows 2000 Server with
> 512 physical memory to a newer Windows 2003 Server with 1024MB physical
> memory.
> The new server functions OK, but there is a very high usage of memory in
> comparison to the old server: the sqlservr.exe process used around 400MB
of
> memory,
> while the new one is using near 850MB after only 2 weeks (while at the
> beginning it used something like 600 MB).
> Is such a behaviour is normal and can be explained by that the new server
> has larger available amount of physical memory?
> In any case, are there any tools that helps to determine which application
> catch the largest amount of memory used by SQL?
> I think that such a large usage for relatively no so hard-working server
is
> something strange.
> Any help/tips will be highly appreciated!!!
> Thanks in advance,
> Barak
>
|||Barak,
BTW, within limits, the more memory available to SQL Server 2000, the better
one can expect performance from SQL Server 2000. Additional memory allows
SQL Server 2000 to create and use a larger memory cache area, thus reducing
physical I/O ... and physical I/O is more often then not what drags down
performance in most any database.
Chief Tenaya
"Barak Turovsky" <baraktur@.mail.ru> wrote in message
news:uslssSIHEHA.1432@.TK2MSFTNGP12.phx.gbl...
> Hi,
> Recently i've moved some of our databases from a Windows 2000 Server with
> 512 physical memory to a newer Windows 2003 Server with 1024MB physical
> memory.
> The new server functions OK, but there is a very high usage of memory in
> comparison to the old server: the sqlservr.exe process used around 400MB
of
> memory,
> while the new one is using near 850MB after only 2 weeks (while at the
> beginning it used something like 600 MB).
> Is such a behaviour is normal and can be explained by that the new server
> has larger available amount of physical memory?
> In any case, are there any tools that helps to determine which application
> catch the largest amount of memory used by SQL?
> I think that such a large usage for relatively no so hard-working server
is
> something strange.
> Any help/tips will be highly appreciated!!!
> Thanks in advance,
> Barak
>

High Availability with Log Shipping

I have implemented log shipping between 2 databases, the transaction log size normally between 10mb - 50mb every 15 minutes during normal working hours, but it grows to 9GB when we run database optimization job and that makes it hard and long to transfer and apply the transaction log on the other database. Does anybody encountered a situation like this and is there a way to minimize the size of the Trans log after the optimization job?

ThanxUse the shrink command: You can look it up. It allow for a backup of the log then it's shrunk to the specified size. Please look up on Book on line or ms sql server support. I suggest shrinking before the other processes.

BACKUP LOG wslogdb62 WITH TRUNCATE_ONLY
DBCC SHRINKFILE('wslogdb62_log.ldf', 1)

High availability at subscribers using database mirroring

I have a mirrored database with transactional replication publishing
to 2 databases on different servers.
The amount of data written to the principal is fairly low.
The requirement is that at least one of the subscribers must be
available at all times with a latency of no greater than five minutes.
I thus have redundancy at the publisher via mirroring and at the
subscribers via multiple databases containing the same data, but I
can't see how to achieve redundancy at the distributor, since the
distribution database cannot be mirrored.
What is the recommended method of making the distributor highly
available? At this site there is reluctance to pursue SQL clustering
and to rename servers as described in BOL. At the moment it looks like
a need a set of scripts to completely set up replication from scratch
if the machine where the distributor DB lives goes offline.
Or, will Katmai offer new features to make all of the replication
components as highly available as the publisher can now be via
mirroring?
Hi Garry,
I understand that you would like to implement a high availability SQL
Server replication. You implemented Database mirroring for your principal
server and multiple databases with same data at the subscribers, but you
would like to know how to implement redundancy at the distributor since the
distribution database cannot be mirrored.
If I have misunderstood, please let me know.
I recommend that you can create a failover cluster for your distributer.
Deploy the distribution database to the virtual server instance. Also, for
your scenario, I think that it is better to use failover cluster for your
publisher since it just need two servers, but to implement a high
availability database mirroring, it needs three servers, principal server,
witness server and mirror server.
For how to setup a SQL Server 2005 failover cluster, you may refer to:
How to: Create a New SQL Server 2005 Failover Cluster (Setup)
http://msdn2.microsoft.com/en-us/library/ms179530.aspx
Hope this helps. If you have any other questions or concerns, please feel
free to let me know.
Have a nice day!
Best regards,
Charles Wang
Microsoft Online Community Support
================================================== ===
Get notification to my posts through email? Please refer to:
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications
If you are using Outlook Express, please make sure you clear the check box
"Tools/Options/Read: Get 300 headers at a time" to see your reply promptly.
Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
================================================== ====
When responding to posts, please "Reply to Group" via
your newsreader so that others may learn and benefit
from this issue.
================================================== ====
This posting is provided "AS IS" with no warranties, and confers no rights.
================================================== ====

HIding databases that a user does not have permission to

I remember reading a blog post which showed you how to hide databases other than the ones the current user has permission to see. My ISP is getting a lot more SQL 2005 databases on the server and I would like to encourage them to only show me mine in Management Studio rather than the huge list I have to scroll down. Is this possible?

(the blog post *may* have been for SQL 2000 but I'm looking for a SQL 2005 solution)

Yes, there have been some recent discussions about this. You need to revoke the permission 'VIEW ANY DATABASE' from the role PUBLIC (and then grant it to any server principal who should have it but don't get it by default). Credit to Kalen Delaney and Dan Guzman who pointed this out... wrote in message news:b45aadf8-59d9-4531-9ba7-dac473bbcade@.discussions.microsoft.com... >I remember reading a blog post which showed you how to hide databases
> other than the ones the current user has permission to see. My ISP is
> getting a lot more SQL 2005 databases on the server and I would like to
> encourage them to only show me mine in Management Studio rather than the
> huge list I have to scroll down. Is this possible? >
> (the blog post *may* have been for SQL 2000 but I'm looking for a SQL
> 2005 solution) > > >
>