Hello,
A new report for accessible MS-RPC will begin distribution tonight.
Please let me know if the sample schema mapping below is acceptable or if any changes are needed.
Regards,
Jason
--
"scan_msrpc" : { "constant_fields" : { "classification.identifier" : "accessible-msrpc", "classification.taxonomy" : "vulnerable", "classification.type" : "vulnerable-system" }, "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", "file_name" : "scan_msrpc", "optional_fields" : [ [ "extra.", "packet_type_value", "convert_int" ], [ "extra.", "fragment_length", "convert_int" ], [ "extra.", "max_transmit", "convert_int" ], [ "extra.", "max_receive", "convert_int" ], [ "extra.", "severity", "validate_to_none" ], [ "protocol.transport", "protocol" ], [ "source.reverse_dns", "hostname" ], [ "extra.", "tag", "validate_to_none" ], [ "source.asn", "asn", "invalidate_zero" ], [ "source.geolocation.cc", "geo" ], [ "source.geolocation.region", "region" ], [ "source.geolocation.city", "city" ], [ "extra.source.naics", "naics", "invalidate_zero" ], [ "extra.", "hostname_source", "validate_to_none" ], [ "extra.source.sector", "sector", "validate_to_none" ], [ "extra.", "version", "validate_to_none" ], [ "extra.", "packet_type", "validate_to_none" ], [ "extra.", "packet_flags", "validate_to_none" ], [ "extra.", "data_representation", "validate_to_none" ], [ "extra.", "auth_length", "validate_to_none" ], [ "extra.", "call_id", "validate_to_none" ], [ "extra.", "association_group", "validate_to_none" ], [ "extra.", "raw_response", "validate_to_none" ] ], "required_fields" : [ [ "time.source", "timestamp", "add_UTC_to_timestamp" ], [ "source.ip", "ip", "validate_ip" ], [ "source.port", "port", "convert_int" ] ], "url" : "https://www.shadowserver.org/what-we-do/network-reporting/what-we-do/network..." }
Hi Jason,
Thank you for notifying us about this. One additional request though: could you please also enable access to the feed's web page? Accessing https://www.shadowserver.org/what-we-do/network-reporting/what-we-do/network... now gives (at least to me) "404 page not found".
Br, Mika
----- Original Message ----- From: "elsif" elsif@shadowserver.org To: "intelmq-dev" intelmq-dev@lists.cert.at Sent: Monday, 2 December, 2024 21:36:48 Subject: [IntelMQ-dev] RFC: scan_msrpc report
Hello,
A new report for accessible MS-RPC will begin distribution tonight.
Please let me know if the sample schema mapping below is acceptable or if any changes are needed.
Regards,
Jason
--
"scan_msrpc" : { "constant_fields" : { "classification.identifier" : "accessible-msrpc", "classification.taxonomy" : "vulnerable", "classification.type" : "vulnerable-system" }, "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", "file_name" : "scan_msrpc", "optional_fields" : [ [ "extra.", "packet_type_value", "convert_int" ], [ "extra.", "fragment_length", "convert_int" ], [ "extra.", "max_transmit", "convert_int" ], [ "extra.", "max_receive", "convert_int" ], [ "extra.", "severity", "validate_to_none" ], [ "protocol.transport", "protocol" ], [ "source.reverse_dns", "hostname" ], [ "extra.", "tag", "validate_to_none" ], [ "source.asn", "asn", "invalidate_zero" ], [ "source.geolocation.cc", "geo" ], [ "source.geolocation.region", "region" ], [ "source.geolocation.city", "city" ], [ "extra.source.naics", "naics", "invalidate_zero" ], [ "extra.", "hostname_source", "validate_to_none" ], [ "extra.source.sector", "sector", "validate_to_none" ], [ "extra.", "version", "validate_to_none" ], [ "extra.", "packet_type", "validate_to_none" ], [ "extra.", "packet_flags", "validate_to_none" ], [ "extra.", "data_representation", "validate_to_none" ], [ "extra.", "auth_length", "validate_to_none" ], [ "extra.", "call_id", "validate_to_none" ], [ "extra.", "association_group", "validate_to_none" ], [ "extra.", "raw_response", "validate_to_none" ] ], "required_fields" : [ [ "time.source", "timestamp", "add_UTC_to_timestamp" ], [ "source.ip", "ip", "validate_ip" ], [ "source.port", "port", "convert_int" ] ], "url" : "https://www.shadowserver.org/what-we-do/network-reporting/what-we-do/network..." }
_______________________________________________ IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
Hi all, thanks for the info.
From my side:
1. The link looks broken, should be https://www.shadowserver.org/what-we-do/network-reporting/ms-rpc-endpoint-ma... 2. As it doesn't assess any vulnerability, I'd suggest the classification type "potentially-unwanted-accessible", what do you think?
The rest looks good to me, thanks for the new report!
Best regards
// Kamil Mańkowski mankowski@cert.at - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien
On 12/3/24 08:00, Mika Silander via IntelMQ-dev wrote:
Hi Jason,
Thank you for notifying us about this. One additional request though: could you please also enable access to the feed's web page? Accessing https://www.shadowserver.org/what-we-do/network-reporting/what-we-do/network... now gives (at least to me) "404 page not found".
Br, Mika
----- Original Message ----- From: "elsif" elsif@shadowserver.org To: "intelmq-dev" intelmq-dev@lists.cert.at Sent: Monday, 2 December, 2024 21:36:48 Subject: [IntelMQ-dev] RFC: scan_msrpc report
Hello,
A new report for accessible MS-RPC will begin distribution tonight.
Please let me know if the sample schema mapping below is acceptable or if any changes are needed.
Regards,
Jason
--
"scan_msrpc" : { "constant_fields" : { "classification.identifier" : "accessible-msrpc", "classification.taxonomy" : "vulnerable", "classification.type" : "vulnerable-system" }, "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", "file_name" : "scan_msrpc", "optional_fields" : [ [ "extra.", "packet_type_value", "convert_int" ], [ "extra.", "fragment_length", "convert_int" ], [ "extra.", "max_transmit", "convert_int" ], [ "extra.", "max_receive", "convert_int" ], [ "extra.", "severity", "validate_to_none" ], [ "protocol.transport", "protocol" ], [ "source.reverse_dns", "hostname" ], [ "extra.", "tag", "validate_to_none" ], [ "source.asn", "asn", "invalidate_zero" ], [ "source.geolocation.cc", "geo" ], [ "source.geolocation.region", "region" ], [ "source.geolocation.city", "city" ], [ "extra.source.naics", "naics", "invalidate_zero" ], [ "extra.", "hostname_source", "validate_to_none" ], [ "extra.source.sector", "sector", "validate_to_none" ], [ "extra.", "version", "validate_to_none" ], [ "extra.", "packet_type", "validate_to_none" ], [ "extra.", "packet_flags", "validate_to_none" ], [ "extra.", "data_representation", "validate_to_none" ], [ "extra.", "auth_length", "validate_to_none" ], [ "extra.", "call_id", "validate_to_none" ], [ "extra.", "association_group", "validate_to_none" ], [ "extra.", "raw_response", "validate_to_none" ] ], "required_fields" : [ [ "time.source", "timestamp", "add_UTC_to_timestamp" ], [ "source.ip", "ip", "validate_ip" ], [ "source.port", "port", "convert_int" ] ], "url" : "https://www.shadowserver.org/what-we-do/network-reporting/what-we-do/network..." }
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/ _______________________________________________ IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
Good morning
On 12/2/24 8:36 PM, elsif wrote:
[ "extra.", "version", "validate_to_none" ],
What "version" is this?
The version of the event specification? The version of the feed? The server version of RPC? The RPC protocol version?
If I'd read just "extra.version" in the event data either as data receiver or operator, I'd have no idea what version is meant here.
best regards Sebastian
Based on the documentation:
"The [major].[minor] version of the MSRPC protocol",
do maybe let's call it "extra.msrpc_version"?
Best regards
// Kamil Mańkowski mankowski@cert.at - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien
On 12/3/24 09:33, Sebix wrote:
Good morning
On 12/2/24 8:36 PM, elsif wrote:
[ "extra.", "version", "validate_to_none" ],
What "version" is this?
The version of the event specification? The version of the feed? The server version of RPC? The RPC protocol version?
If I'd read just "extra.version" in the event data either as data receiver or operator, I'd have no idea what version is meant here.
best regards Sebastian
Hi Kamil
On 12/3/24 9:37 AM, Kamil Mankowski via IntelMQ-dev wrote:
Based on the documentation:
"The [major].[minor] version of the MSRPC protocol",
do maybe let's call it "extra.msrpc_version"?
Yes, I think that is intuitively understandable. Thanks for the proposal.
Sebastian
Thank you for your comments.
Here are the changes based on your feedback:
"scan_msrpc" : { "constant_fields" : { "classification.identifier" : "accessible-msrpc", "classification.taxonomy" : "vulnerable", "classification.type" : "potentially-unwanted-accessible" }, "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", "file_name" : "scan_msrpc", "optional_fields" : [ [ "extra.msrpc_version", "version", "convert_float" ],
...
"url" : "https://www.shadowserver.org/what-we-do/network-reporting/ms-rpc-endpoint-ma..."
}
Please let me if that know if any changes are needed or it is ready to publish.
Regards,
Jason
It looks good to me :)
Best regards
// Kamil Mańkowski mankowski@cert.at - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien
On 12/3/24 17:25, elsif wrote:
Thank you for your comments.
Here are the changes based on your feedback:
"scan_msrpc" : { "constant_fields" : { "classification.identifier" : "accessible-msrpc", "classification.taxonomy" : "vulnerable", "classification.type" : "potentially-unwanted-accessible" }, "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", "file_name" : "scan_msrpc", "optional_fields" : [ [ "extra.msrpc_version", "version", "convert_float" ],
...
"url" : "https://www.shadowserver.org/what-we-do/network-reporting/ms-rpc-endpoint-ma..."
}
Please let me if that know if any changes are needed or it is ready to publish.
Regards,
Jason
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
Thank you. The schema update has been published.
Regards,
Jason
On 12/5/24 4:42 AM, Kamil Mankowski via IntelMQ-dev wrote:
It looks good to me :)
Best regards
// Kamil Mańkowski mankowski@cert.at - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien
On 12/3/24 17:25, elsif wrote:
Thank you for your comments.
Here are the changes based on your feedback:
"scan_msrpc" : { "constant_fields" : { "classification.identifier" : "accessible-msrpc", "classification.taxonomy" : "vulnerable", "classification.type" : "potentially-unwanted-accessible" }, "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", "file_name" : "scan_msrpc", "optional_fields" : [ [ "extra.msrpc_version", "version", "convert_float" ],
...
"url" : "https://www.shadowserver.org/what-we-do/network-reporting/ms-rpc-endpoint-ma..."
}
Please let me if that know if any changes are needed or it is ready to publish.
Regards,
Jason
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
Hi all,
sorry for jumping in late... I was out of office last week.
Kamil Mankowski wrote:
- As it doesn't assess any vulnerability, I'd suggest the classification type "potentially-unwanted-accessible", what do you think?
This is true for many other open-* and accessible-* reports as well. We discussed this with the first version of the schema and decided to stay with "vulnerable-service" and change the classification type to "potentially-unwanted-accessible" where appropriate for all reports at once at some time later to not mix up things.
scan_msrpc now is the only report with classification.type "potentially-unwanted-accessible".
I'd suggest setting the classification type to "vulnerable-service" here for now and change it to "potentially-unwanted-accessible" at some time later along with all other reports where appropriate.
Sebix wrote:
If I'd read just "extra.version" in the event data either as data receiver or operator, I'd have no idea what version is meant here.
We have "extra.version" with many other reports like open-elasticsearch, accessible-activemq or accessible-mysql as well.
I think that in a "scan_msrpc" report, it's intuitive that "version" is the msrpc version.
We usually map all extra fields from the reports using their original name (like "version" -> "extra.version" or "tag" -> "extra.tag").
What we IMHO should NOT do is breaking this convention by mapping "version" to "extra.msrpc_version".
So I'd suggest keeping "extra.version" like with other reports.
Regards Thomas
On 05.12.24 17:27, elsif wrote:
Thank you. The schema update has been published.
Regards,
Jason
On 12/5/24 4:42 AM, Kamil Mankowski via IntelMQ-dev wrote:
It looks good to me :)
Best regards
// Kamil Mańkowski mankowski@cert.at - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien
On 12/3/24 17:25, elsif wrote:
Thank you for your comments.
Here are the changes based on your feedback:
"scan_msrpc" : { "constant_fields" : { "classification.identifier" : "accessible-msrpc", "classification.taxonomy" : "vulnerable", "classification.type" : "potentially-unwanted-accessible" }, "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", "file_name" : "scan_msrpc", "optional_fields" : [ [ "extra.msrpc_version", "version", "convert_float" ],
...
"url" : "https://www.shadowserver.org/what-we-do/network-reporting/ms-rpc-endpoint-ma..."
}
Please let me if that know if any changes are needed or it is ready to publish.
Regards,
Jason
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
Morning!
I'd like to pick up this thread again.
We had always kept the original field names (for "version" and others) from the Shadowserver reports when writing to extra so there was a 1:1 mapping.
Now we broke this convention by writing "version" to "extra.msrpc_version" which makes things prone to errors when referencing fields.
So I'd like to suggest changing this from
"extra.msrpc_version", "version", "convert_float"
to
"extra.version", "version", "convert_float"
again.
For the classification type, I'm fine with keeping this set to "potentially-unwanted-accessible" if we change the classification type for other events like accessible-postgresql from "vulnerable-system" to "potentially-unwanted-accessible" in a timely manner as well.
What do you think?
Regards Thomas
On 09.12.24 15:43, Thomas Hungenberg via IntelMQ-dev wrote:
Hi all,
sorry for jumping in late... I was out of office last week.
Kamil Mankowski wrote:
- As it doesn't assess any vulnerability, I'd suggest the classification type "potentially-unwanted-accessible", what do you think?
This is true for many other open-* and accessible-* reports as well. We discussed this with the first version of the schema and decided to stay with "vulnerable-service" and change the classification type to "potentially-unwanted-accessible" where appropriate for all reports at once at some time later to not mix up things.
scan_msrpc now is the only report with classification.type "potentially-unwanted-accessible".
I'd suggest setting the classification type to "vulnerable-service" here for now and change it to "potentially-unwanted-accessible" at some time later along with all other reports where appropriate.
Sebix wrote:
If I'd read just "extra.version" in the event data either as data receiver or operator, I'd have no idea what version is meant here.
We have "extra.version" with many other reports like open-elasticsearch, accessible-activemq or accessible-mysql as well.
I think that in a "scan_msrpc" report, it's intuitive that "version" is the msrpc version.
We usually map all extra fields from the reports using their original name (like "version" -> "extra.version" or "tag" -> "extra.tag").
What we IMHO should NOT do is breaking this convention by mapping "version" to "extra.msrpc_version".
So I'd suggest keeping "extra.version" like with other reports.
Regards Thomas
On 05.12.24 17:27, elsif wrote:
Thank you. The schema update has been published.
Regards,
Jason
On 12/5/24 4:42 AM, Kamil Mankowski via IntelMQ-dev wrote:
It looks good to me :)
Best regards
// Kamil Mańkowski mankowski@cert.at - T: +43 676 898 298 7204 // CERT Austria - https://www.cert.at/ // CERT.at GmbH, FB-Nr. 561772k, HG Wien
On 12/3/24 17:25, elsif wrote:
Thank you for your comments.
Here are the changes based on your feedback:
"scan_msrpc" : { "constant_fields" : { "classification.identifier" : "accessible-msrpc", "classification.taxonomy" : "vulnerable", "classification.type" : "potentially-unwanted-accessible" }, "feed_name" : "Accessible-MS-RPC-Endpoint-Mapper", "file_name" : "scan_msrpc", "optional_fields" : [ [ "extra.msrpc_version", "version", "convert_float" ],
...
"url" : "https://www.shadowserver.org/what-we-do/network-reporting/ms-rpc-endpoint-ma..."
}
Please let me if that know if any changes are needed or it is ready to publish.
Regards,
Jason
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
IntelMQ-dev mailing list https://lists.cert.at/cgi-bin/mailman/listinfo/intelmq-dev https://docs.intelmq.org/
Hi Thomas,
On 1/7/25 10:37 AM, Thomas Hungenberg wrote:
We had always kept the original field names (for "version" and others) from the Shadowserver reports when writing to extra so there was a 1:1 mapping.
Not always, but mostly. In most cases, the data field names of Shadowserver are good enough and appropriate to use them 1:1. I don't know of any convention that we must use them.
Now we broke this convention by writing "version" to "extra.msrpc_version" which makes things prone to errors when referencing fields.
As I wrote before, "version" is ambiguous for me, it's unclear what the version field refers to.
It could be the version of the IntelMQ data format. It could be the version of the IntelMQ software. It could be the version of the data feed. It could be the version of the server software. And it could be the version an application protocol, as it is in this case.
just my 2 cents
Sebastian
Hi Sebastian,
On 07.01.25 10:53, Sebix wrote:
As I wrote before, "version" is ambiguous for me, it's unclear what the version field refers to.
It could be the version of the IntelMQ data format. It could be the version of the IntelMQ software. It could be the version of the data feed. It could be the version of the server software. And it could be the version an application protocol, as it is in this case.
I don't agree here. We have been using "version" for a long time with many other reports like Open-Elasticsearch, Vulnerable-Exchange, Accessible-ActiveMQ or Accessible-PostgreSQL and it's always referring to the application version.
So IMHO one can intuitively assume that in this case "version" corresponds to the version of the application reported with this event and not the version of the IntelMQ software or data format.
Regards Thomas