Showing posts with label folks. Show all posts
Showing posts with label folks. Show all posts

Monday, March 26, 2012

Historical Stats

Ok folks,
Need your help once again.

I have 4 Tables that need to get accurate stats from...

2 are historical, 2 are current

the 2 historical are identical to the 2 present ones, just the historical is everything up to a cutoff date, the the 2 present is everything from that cutoff date on...

They contain login information, access times, install counts etc.

eg. OldTable1 = serialNo, ID, Install Date, NoOfInstalls, etc..
OldTable2 = SerailNo, Accessdate, Installid, UserID etc.
NewTable1 = serialNo, ID, Install Date, NoOfInstalls, etc..
NewTable2 = SerailNo, Accessdate, Installid, UserID etc.
I need to get total installs (gathered from both tables), total accesses after given dates etc...

the records are created at each login, so records may be in both sets of tables, (ie, if accessed after cutoff, a new record is created in the new tables...)

Any ideas?

Thanks in advanceLook up the use of the UNION operator in Books Online. It can be used to effectively combine two tables for processing, or to combine the results of two select statements.

blindman|||And if you post some DDL, DML and sample data, I bet we can help alot more..

CREATE TABEL myTable99 (col1 int, ect

INSERT INTO myTable99 (col1, ect
SELECT 1, ect UNION ALL
SELECT 1, ect UNION ALL
SELECT 1, ect UNION ALL
SELECT 1, ect

SELECT * FROM myTable99 WHERE ect -- or whatever you've tried som far...

might be worth your time...|||This is what I am trying, but it has not been working..I get this is not permitted when the subquery follows ... (or when the subquery follows and expression)

I am trying to do this with out a new table, but a view is fine...

I am just trying to count version 4s from old, version 4s from new, then the union that combines (just as a test audit for now)

should be minimal overhead this way...

CREATE VIEW dbo.FV_Serial_Reg
AS

SELECT COUNT(SerialNumber) AS Ver4_Old,
(

SELECT COUNT(SerialNumber)
FROM FRV.dbo.WFSerialNumber
WHERE left(Version,1) = '4'
) AS Ver4_New,
(
SELECT Version4 = COUNT(SerialNumber) FROM FRV.dbo.WFSerialNumber WHERE left(Version,1) = '4'
UNION
SELECT COUNT(SerialNumber) FROM FRV.dbo.WFSerialNumber_PREV WHERE left(Version,1) = '4'
) AS Version4


FROM FRV.dbo.WFSerialNumber_Prev
WHERE left(Version,1) = '4'|||DUDE.....

ya lost me...is there a question here?

why are you doing what you're doing?

Does it work|||I mentioned the error in the previous post...

This is what I am trying, but it has not been working..

I get this is not permitted when the subquery follows ... (or when the subquery follows and expression)

I need these stats in a view so that we access them via a data access page created with MS Access.

Friday, March 23, 2012

High performance system and table and index structure

Hi Folks,
I'm working on something a little different at present. Given it's a bit out
of the ordinary, and it relates to some of our indexing discussions lately,
I'd appreciate any thoughts/feedback.
It's a SQL Server box that's receiving a stream of real-time data from oil
drilling rigs all over the world. It needs to perform both fast inserts (ie
10,000 per second) and also have good performance on a fixed list of query
types. The essential part of the table data that has me most intrigued looks
like this:
Well (ID of the well involved - varchar(60))
Wellbore (ID of a specific bore at a wellsite - varchar(60))
Curve (ID of a specific set of measures eg: pressures, gas levels, etc.)
Depth (float)
Time (datetime)
DataValue (float)
This data is defined in an external xsd. The hardware/operating system
requirements are basically a non-issue ie: whatever is needed can be
obtained.
The key thing is that about 100 wellbores with up to 100 curves (of sets of
data) can be active each second, so fast insert performance is critical.
Once inserted, the messiest queries involve:
1. All values for a specific curve within a range of depths
2. All values for a specific curve within a range of times
The things I'm pondering are:
1. Whether to have this in one database or to have a database per well.
There are about 1200 wells, but about 100 active concurrently.
2. Whether to store the data in a heap or a clustered index on either depth
or time.
3. Whether to use a non-clustered index for whichever of depth or time
wasn't the clustered one OR whether to store the data twice ie: once by date
and once by time.
4. Whether to find a way to limit the range of depth values (currently
float) to store them instead as a bigint, if the performance of bigint's in
indexes and range queries, etc. is better than floats.
I'd love to hear anyone's thoughts on this. We're about to start
experimenting to see what works.
Regards,
GregGreg
Is it SQL Server 2005?
Well , if it needs perfom fast inserts I think to store the data in a
heap you can get better performance for inserting , however as you said
you need to have good performance on lits of query , so CI will be a good
candidate for range queries
Actually it is really hard to suggest something , you need to test it and
get a decision
"Greg Low [MVP]" <greglow@.lowell.com.au> wrote in message
news:udZGEsj9GHA.3264@.TK2MSFTNGP04.phx.gbl...
> Hi Folks,
> I'm working on something a little different at present. Given it's a bit
> out of the ordinary, and it relates to some of our indexing discussions
> lately, I'd appreciate any thoughts/feedback.
> It's a SQL Server box that's receiving a stream of real-time data from oil
> drilling rigs all over the world. It needs to perform both fast inserts
> (ie 10,000 per second) and also have good performance on a fixed list of
> query types. The essential part of the table data that has me most
> intrigued looks like this:
> Well (ID of the well involved - varchar(60))
> Wellbore (ID of a specific bore at a wellsite - varchar(60))
> Curve (ID of a specific set of measures eg: pressures, gas levels, etc.)
> Depth (float)
> Time (datetime)
> DataValue (float)
> This data is defined in an external xsd. The hardware/operating system
> requirements are basically a non-issue ie: whatever is needed can be
> obtained.
> The key thing is that about 100 wellbores with up to 100 curves (of sets
> of data) can be active each second, so fast insert performance is
> critical. Once inserted, the messiest queries involve:
> 1. All values for a specific curve within a range of depths
> 2. All values for a specific curve within a range of times
> The things I'm pondering are:
> 1. Whether to have this in one database or to have a database per well.
> There are about 1200 wells, but about 100 active concurrently.
> 2. Whether to store the data in a heap or a clustered index on either
> depth or time.
> 3. Whether to use a non-clustered index for whichever of depth or time
> wasn't the clustered one OR whether to store the data twice ie: once by
> date and once by time.
> 4. Whether to find a way to limit the range of depth values (currently
> float) to store them instead as a bigint, if the performance of bigint's
> in indexes and range queries, etc. is better than floats.
> I'd love to hear anyone's thoughts on this. We're about to start
> experimenting to see what works.
> Regards,
> Greg
>|||Hi...
We had a simmilar problem just alitle bit scaled down.
We implemented like:
- use an autoid for PK -> heap, rows are appended in the pages, no other
indexes, no constrains, all collumns allows null.
- use EE version (sql2005) :)
- use snapshot isolation mode for querying the data. (decrease lockings)
- use for querying a second maschine wich is sync-ing the data each n
sec/min. (you can try a log shipping but we've got no time to checkit)
(decreasing lockings, free cpu).
Have phun,
Radu
"Greg Low [MVP]" <greglow@.lowell.com.au> wrote in message
news:udZGEsj9GHA.3264@.TK2MSFTNGP04.phx.gbl...
> Hi Folks,
> I'm working on something a little different at present. Given it's a bit
> out of the ordinary, and it relates to some of our indexing discussions
> lately, I'd appreciate any thoughts/feedback.
> It's a SQL Server box that's receiving a stream of real-time data from oil
> drilling rigs all over the world. It needs to perform both fast inserts
> (ie 10,000 per second) and also have good performance on a fixed list of
> query types. The essential part of the table data that has me most
> intrigued looks like this:
> Well (ID of the well involved - varchar(60))
> Wellbore (ID of a specific bore at a wellsite - varchar(60))
> Curve (ID of a specific set of measures eg: pressures, gas levels, etc.)
> Depth (float)
> Time (datetime)
> DataValue (float)
> This data is defined in an external xsd. The hardware/operating system
> requirements are basically a non-issue ie: whatever is needed can be
> obtained.
> The key thing is that about 100 wellbores with up to 100 curves (of sets
> of data) can be active each second, so fast insert performance is
> critical. Once inserted, the messiest queries involve:
> 1. All values for a specific curve within a range of depths
> 2. All values for a specific curve within a range of times
> The things I'm pondering are:
> 1. Whether to have this in one database or to have a database per well.
> There are about 1200 wells, but about 100 active concurrently.
> 2. Whether to store the data in a heap or a clustered index on either
> depth or time.
> 3. Whether to use a non-clustered index for whichever of depth or time
> wasn't the clustered one OR whether to store the data twice ie: once by
> date and once by time.
> 4. Whether to find a way to limit the range of depth values (currently
> float) to store them instead as a bigint, if the performance of bigint's
> in indexes and range queries, etc. is better than floats.
> I'd love to hear anyone's thoughts on this. We're about to start
> experimenting to see what works.
> Regards,
> Greg
>|||On 23.10.2006 02:34, Greg Low [MVP] wrote:
> Hi Folks,
> I'm working on something a little different at present. Given it's a bit out
> of the ordinary, and it relates to some of our indexing discussions lately,
> I'd appreciate any thoughts/feedback.
> It's a SQL Server box that's receiving a stream of real-time data from oil
> drilling rigs all over the world. It needs to perform both fast inserts (ie
> 10,000 per second) and also have good performance on a fixed list of query
> types. The essential part of the table data that has me most intrigued looks
> like this:
> Well (ID of the well involved - varchar(60))
> Wellbore (ID of a specific bore at a wellsite - varchar(60))
> Curve (ID of a specific set of measures eg: pressures, gas levels, etc.)
> Depth (float)
> Time (datetime)
> DataValue (float)
> This data is defined in an external xsd. The hardware/operating system
> requirements are basically a non-issue ie: whatever is needed can be
> obtained.
> The key thing is that about 100 wellbores with up to 100 curves (of sets of
> data) can be active each second, so fast insert performance is critical.
> Once inserted, the messiest queries involve:
> 1. All values for a specific curve within a range of depths
> 2. All values for a specific curve within a range of times
> The things I'm pondering are:
> 1. Whether to have this in one database or to have a database per well.
> There are about 1200 wells, but about 100 active concurrently.
What about partitioning by wellid? You should at least create a setup
with allows for as much concurrent processing as possible (i.e.
separating data physically on different disks or at least file groups
etc.) to be able to stream your data into the DB as independently as
possible.
> 2. Whether to store the data in a heap or a clustered index on either depth
> or time.
> 3. Whether to use a non-clustered index for whichever of depth or time
> wasn't the clustered one OR whether to store the data twice ie: once by date
> and once by time.
If you need purging of old data and query according to time range then a
CI with timestamp as leading column is certainly a good idea.
> 4. Whether to find a way to limit the range of depth values (currently
> float) to store them instead as a bigint, if the performance of bigint's in
> indexes and range queries, etc. is better than floats.
I would start out with floats if the data is actually float. Only go
for this type of optimization if you actually need it and it is proved
that bigint is faster.
> I'd love to hear anyone's thoughts on this. We're about to start
> experimenting to see what works.
On a more general level I am thinking that Oracle might be more suited
for this scenario because it offers several features that give you
better control. For example, you do not have a single CI on a table -
basically all indexes are ordered (but you can mimic SQL Server's CI
with an IOT). Another point is partitioning of tables, Oracle is much
more flexible there (IIRC you can do only hash partitioning on SQL 2005).
Just my 0.02EUR...
Kind regards
robert

Sunday, February 19, 2012

Hiding a Subreport

Hi, Folks.
I have a "List" report which contains a "Table"-style subreport.
I'd like to display the table, WITH the headers, as the subreport ONLY IF
there are some elements to be displayed within the table. In many cases,
there will be no data in the subreport. I'd like to suppress the entire
subreport if there are no rows in the table.
Alternatively, I'd be happy to simply suppress/hide the table header row
when the number of rows returned is zero. I have a field in the table
header called "NumKey", which is the Count(KeyValue). If no records, then
the table should be hidden; if there are one or more records, all the
records AND the table leader should be displayed.
Alas, there doesn't seem to be anyplace that I can access the "Visible"
property for the entire subreport from within the subreport itself, and
there's no place within the table header that allows me to change the
"Visible" propery of the header row based on the "NumKey" text box.
This ought to be possible. What am I missing?
Thanks.
--
---
Ken Mitchell ken@.creativemindssacramento.com
916-275-3610 (voice) Citrus Heights, CA
--http://www.creativemindssacramento.com--
"Those of us who DO remember our history are condemned to live
watching others around us trying to repeat it. "
---Two options:
1. Set the visible property on the table and use Count(KeyValue) in the
expression for this property.
2. Set the NoRows property on the table, so the NoRows text will be shown
instead of the table when there is no data.
Fang Wang (MSFT)
Microsoft SQL Server Reporting Services
This posting is provided "AS IS" with no warranties, and confers no rights.
"Ken Mitchell" <cmmcken@.comcast.net> wrote in message
news:OFTN4w%23XEHA.2972@.TK2MSFTNGP12.phx.gbl...
> Hi, Folks.
> I have a "List" report which contains a "Table"-style subreport.
> I'd like to display the table, WITH the headers, as the subreport ONLY IF
> there are some elements to be displayed within the table. In many cases,
> there will be no data in the subreport. I'd like to suppress the entire
> subreport if there are no rows in the table.
> Alternatively, I'd be happy to simply suppress/hide the table header row
> when the number of rows returned is zero. I have a field in the table
> header called "NumKey", which is the Count(KeyValue). If no records, then
> the table should be hidden; if there are one or more records, all the
> records AND the table leader should be displayed.
> Alas, there doesn't seem to be anyplace that I can access the "Visible"
> property for the entire subreport from within the subreport itself, and
> there's no place within the table header that allows me to change the
> "Visible" propery of the header row based on the "NumKey" text box.
> This ought to be possible. What am I missing?
> Thanks.
> --
> ---
> Ken Mitchell ken@.creativemindssacramento.com
> 916-275-3610 (voice) Citrus Heights, CA
> --http://www.creativemindssacramento.com--
> "Those of us who DO remember our history are condemned to live
> watching others around us trying to repeat it. "
> ---
>
>